
PROGRAMMING15. CUSTOM MACRO
B–64134EN/01
252
Errors may occur when operations are performed.
Table 15.3 (b) Errors involved in operations
Operation Average
error
Maximum
error
Type of error
a = b*c 1.55×10
–10
4.66×10
–10
a = b / c 4.66×10
–10
1.88×10
–9
1.24×10
–9
3.73×10
–9
a = b + c
a = b – c
2.33×10
–10
5.32×10
–10
a = SIN [ b ]
a = COS [ b ]
5.0×10
–9
1.0×10
–8
a = ATAN [ b ] / [ c ] (*4) 1.8×10
–6
3.6×10
–6
NOTE
1 The relative error depends on the result of the operation.
2 Smaller of the two types of errors is used.
3 The absolute error is constant, regardless of the result of the
operation.
4 Function TAN performs SIN/COS.
5 Setting parameter No.6004#1 to 1 normalizes the operation
result to 0 when the operation result of function SIN, COS,
or TAN is less than 1.010
–8
or when 0 cannot be obtained
due to the limitation of the precision.
S The precision of variable values is about 8 decimal digits. When very
large numbers are handled in an addition or subtraction, the expected
results may not be obtained.
Example:
When an attempt is made to assign the following values to
variables #1 and #2:
#1=9876543210123.456
#2=9876543277777.777
the values of the variables become:
#1=9876543200000.000
#2=9876543300000.000
In this case, when #3=#2–#1; is calculated, #3=100000.000 results.
(The actual result of this calculation is slightly different because
it is performed in binary.)
D Operation error
a + b
Ǹ
Relative error(*1)
ε
b
Min
(*2)
ε
b
ε
c
ε
Absolute error(*3)
degrees