[babl] babl: decrease tolerance



commit 5e40364b3953f90051e290dd185b91e64aea1a66
Author: Øyvind Kolås <pippin gimp org>
Date:   Tue Nov 27 03:37:10 2018 +0100

    babl: decrease tolerance
    
    The following conversion is undesirable, going via linear u16 on the
    way to a different TRC u16 amount of quantization should be considered
    bad enough to be avoided.
    
    chosen RGBA u32 to R'G'B'A u16: steps: 4 error: 0.000005 cost: 876.000000
            /usr/local/lib/babl-0.1/u32.so 0: RGBA u32 to RGBA u16
            /usr/local/lib/babl-0.1/gggl-table-lies.so 0: RGBA u16 to RGBA float
            /usr/local/lib/babl-0.1/sse2-float.so 0: RGBA float to R'G'B'A float
            /usr/local/lib/babl-0.1/gggl.so 0: R'G'B'A float to R'G'B'A u16
    
    This is slower but more correct:
    
    chosen RGBA u32 to R'G'B'A u16: steps: 3 error: 0.000000 cost: 1051.000000
            /usr/local/lib/babl-0.1/simple.so 0: RGBA u32 to RGBA float
            /usr/local/lib/babl-0.1/sse2-float.so 0: RGBA float to R'G'B'A float
            /usr/local/lib/babl-0.1/gggl.so 0: R'G'B'A float to R'G'B'A u16

 babl/babl-fish-path.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
---
diff --git a/babl/babl-fish-path.c b/babl/babl-fish-path.c
index c3a2289..2524bb4 100644
--- a/babl/babl-fish-path.c
+++ b/babl/babl-fish-path.c
@@ -21,7 +21,7 @@
 #include "babl-internal.h"
 #include "babl-ref-pixels.h"
 
-#define BABL_TOLERANCE             0.000005
+#define BABL_TOLERANCE             0.0000047
 #define BABL_MAX_COST_VALUE        2000000
 #define BABL_HARD_MAX_PATH_LENGTH  8
 #define BABL_MAX_NAME_LEN          1024


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]