Product Feature: Design Studio
While using 'To decimal (16,3)' function in the Expression calculator transform, output appears as 16.236012 rather than converting it to decimal i.e. (16,3) = 16.236. This behavior is observed while comparing two values like To_Decimal (1.3681,38,3) and To_Decimal (1.3680,38,3).
For the sake of performance, decimal values are converted to type double when the values are used in calculations or comparisons. However, this process may result in a loss of precision. To ensure a specific accuracy of decimal results, decimal math functionality is enabled. Transforms that do sorts or comparisons with decimal fields may be affected by enabling decimal math, because decimal values are no longer converted to type double.
ResolutionUPDATED: October 23, 2018
Follow these steps:
- Navigate to Tools->'Plan Properties'
- Enable/choose 'Use decimal math provider'.