In our continuous efforts to enhance performance and resource utilization, we are excited to introduce a new feature aimed at optimizing the Track Changes functionality in ERP.net. With the release of version 24, Track Changes will now require significantly less storage space and CPU power, leading to faster operations and more efficient data management.
One of the driving factors behind this update was the rapid growth of the Sys_Attribute_Changes table, which had become unwieldy in large databases. For example, this table has grown to several terabytes in some databases and contains tens of billions of records. Clearly, a more efficient solution was needed to manage such vast amounts of data without compromising performance.
What’s New?
The primary goal of this update was to reduce the space and time required for tracking changes in your ERP.net database. After extensive testing and development, we have implemented a series of improvements:
- Elimination of Redundant Data Storage:
- The Sys_Attribute_Changes table, which previously stored changes in object attributes, has been fully retired. Instead, all attribute changes are now stored directly in the JSON field within the Sys_Object_Changes table. This not only simplifies data management but also reduces the amount of storage required.
- JSON Compression:
- To further optimize storage, we’ve introduced JSON compression for the Old_Values_Json field. The data is stored in a compressed format, reducing space requirements significantly. In cases where the JSON string exceeds 50 characters, compression results in notable space savings.
- Asynchronous Processing:
- The process of tracking changes has been optimized by making the creation of Sys_Object_Changes asynchronous. This means that rather than creating and updating records synchronously, which can slow down operations, the system now processes these changes in the background, improving overall system responsiveness.
Performance Gains
Our testing has revealed significant gains in both processing speed and storage efficiency with the new update.
Here’s a comparison of the system’s performance when processing 50 Sales Order documents, each containing 6 lines:
Before Version 24
Total Modified Rows (including Sys_Object_Changes): 3410;
Total Time: 108.287 sec;
Attribute Changes Tracking Time: 27.758 sec;
Sys_Object_Changes: 1400 rows; 6.520 MB;
Sys_Attribute_Changes: 13050 rows; 7.008 MB;
In Version 24
Total Modified Rows (including Sys_Object_Changes): 3410;
Total Time: 79.123 sec;
Attribute Changes Tracking Time: 9.904 sec;
Sys_Object_Changes: 1400 rows; 6.208 MB;
Sys_Attribute_Changes: 0 rows; 0.000 MB;
As you can see:
* The overall processing time has decreased by nearly 27%, from 108.287 seconds to 79.123 seconds.
* There is a dramatic reduction in the time spent tracking attribute changes, dropping from 27.758 seconds to just 9.904 seconds—a decrease of approximately 64%.
* The space required for Sys_Object_Changes slightly decreased, from 6.520 MB to 6.208 MB. More significantly, Sys_Attribute_Changes, which previously consumed 7.008 MB, now takes up no additional space, reducing storage usage by over 50%.
What Does This Mean for You?
With these optimizations, you'll experience faster processing times and more efficient use of system resources.
Since Track Changes is a widely used feature, its improvements will enhance the performance of many processes across your database.
To take advantage of these benefits, simply upgrade to version 24.
Looking Ahead
This optimization is just one part of our ongoing efforts to make ERP.net more powerful and efficient. We are committed to continuous improvement and we are thankful for your feedback that helps us improve.
Upgrade to version 24 today and experience the enhanced Track Changes functionality for yourself!
Stay tuned for more updates and improvements in future releases.
0 Comments