[librsvg] tests/README.md: Update for the new test types



commit 0a02f6a60e07167649997bd5fab2edc65942bc66
Author: Federico Mena Quintero <federico gnome org>
Date:   Mon Nov 27 08:24:44 2017 -0600

    tests/README.md: Update for the new test types

 tests/README.md |  147 ++++++++++++++++++++++++++++++++++--------------------
 1 files changed, 92 insertions(+), 55 deletions(-)
---
diff --git a/tests/README.md b/tests/README.md
index 91a741b..fd1f4ba 100644
--- a/tests/README.md
+++ b/tests/README.md
@@ -1,19 +1,30 @@
 Librsvg test suite
 ==================
 
-Librsvg's test harness is built upon Glib's GTest utility functions.
-These let you define tests in the C language.
+Librsvg's test harness is built upon [Glib's GTest utility
+functions][gtest].  These let you define tests in the C language.
 
-There are three different styles of tests:
+There are several different styles of tests:
 
-* crash - Ensures that loading and parsing (but not rendering) a
-  particular SVG doesn't yield a GError.
+* `rsvg-test` - The main test suite; loads, parses, and renders SVGs and
+  compares the results against reference PNG images.
 
-* styles - Loads and parses an SVG, and checks that a particular SVG
-  object has a certain style.
+* `loading` - Tests that compressed and uncompressed SVGs are loaded
+  correctly even if read one or two bytes at a time.  This is
+  basically a test for the state machines in the loading code.
 
-* rsvg-test - The main test suite; loads, parses, and renders SVGs and
-  compares the results against reference PNG images.
+* `crash` - Ensures that loading and parsing (but not rendering) a
+  particular SVG doesn't crash or yield a GError.
+  
+* `render-crash` - Ensures that rendering a loaded SVG doesn't crash.
+
+* `dimensions` - Loads an SVG, and tests that librsvg computes the
+  correct dimensions for the toplevel SVG element, or one of the
+  individual sub-elements.
+
+* `styles` - Loads and parses an SVG, and checks that a particular SVG
+  object has a certain style.  These tests are currently disabled,
+  since they depend on internal APIs of librsvg.
 
 These are all "black box tests": they run the library with its public
 API, and test the results.  They do not test the library's internals;
@@ -31,7 +42,8 @@ directory and run `make check`.  This will run both the small unit
 tests and the black box tests in this `librsvg/tests` directory.
 
 If you want to run just the black box tests, go into this
-`librsvg/tests` directory and run `make check`.
+`librsvg/tests` directory and run `make check`.  If you want to run
+the unit tests, go to `librsvg/rust` and run `cargo test`.
 
 Those commands will yield exit code 0 if all the tests pass, or
 nonzero if some tests fail.
@@ -39,10 +51,9 @@ nonzero if some tests fail.
 Running `make check` will produce a `test/test-suite.log` file.  You can
 see this file for the details of failed tests.
 
-Additionally, all three sets of black box tests (crash, styles,
-rsvg-test) will produce a test report in a text file.  In the tests
-directory, you can see `crash.log`, `styles.log`, `rsvg-test.log`,
-respectively.
+Additionally, all the black box tests (rsvg-test, crash, etc.) will
+produce a test report in a text file.  In the tests directory, you can
+see `rsvg-test.log`, `crash.log`, etc., respectively.
 
 
 # Tests and test fixtures
@@ -56,31 +67,31 @@ Each image-based reference test uses two files: `foo.svg` and
 `foo-ref.png`.  The test harness will render `foo.svg` and compare the
 results to `foo-ref.png`.  Currently we assume a pixel-perfect match.
 If there are differences in the output, the test will fail; see
-"Examining failed reference tests" below.
+"[Examining failed reference tests][#examining-failed-reference-tests]" below.
 
 These files can go anywhere under the `fixtures/reftests`
 directory; the `rsvg-test` program will recursively look inside
 `fixtures/reftests` for all SVG files, render them, and compare them to
 the `-ref.png` reference images.
 
-SVG test files or entire subdirectories in fixtures/reftests whose
-names begin with "ignore" will be skipped from the tests.  That is,
-anything that matches "fixtures/reftests/ignore*" will not be included
-in the tests.  You can use this to skip a few problematic files
-temporarily.
+**Ignoring tests: ** SVG test files or entire subdirectories in
+`fixtures/reftests` whose names begin with "`ignore`" will be skipped from
+the tests.  That is, anything that matches "`fixtures/reftests/ignore*`"
+will not be included in the tests.  You can use this to skip a few
+problematic files temporarily.
 
 As of 2016/Nov/03 we have an informal organization of these files:
 
-* `fixtures/reftests/*.svg` - Tests for special situations
-  that arose during development.
+* `fixtures/reftests/svg1.1` - Tests from the W3C's SVG test suite.
+  These are supposed to test all of SVG's features; we will add them one
+  by one as librsvg starts implementing the features.
 
 * `fixtures/reftests/bugs/*.svg` - Tests for particular bug numbers.
-  Please use the bug number from Bugzilla, like 123456.svg, and the
-  corresponding 123456-ref.png for the known-good reference image.
+  Please use the bug number from Bugzilla, like `123456.svg`, and the
+  corresponding `123456-ref.png` for the known-good reference image.
 
-* `fixtures/reftests/svg1.1` - Tests from the W3C's SVG test suite.
-  These are supposed to test all of SVG's features; we'll add them one
-  by one as librsvg starts implementing the features.
+* `fixtures/reftests/*.svg` - Tests for special situations
+  that arose during development.
 
 ### Examining failed reference tests
 
@@ -101,7 +112,7 @@ This means that the test named
 autogenerated test names from GTest.  In this case, it means that the
 test file `fixtures/reftests/svg1.1/paths-data-18-f.svg` got rendered,
 and produced incorrect output when compared to
-`fixtures/reftests/svg1.1/paths-data-18-f-out.png`.
+`fixtures/reftests/svg1.1/paths-data-18-f-ref.png`.
 
 When a test fails, rsvg-test creates two images in `/tmp`:
 
@@ -121,29 +132,27 @@ It is up to you to decide what to do next:
   with respect to the `foo-ref.png` reference image is that
   antialiased edges look different, or font rendering is slightly
   different due to the font-rendering machinery in your system, you
-  can just regenerate the test image.  See "Regenerating reference
-  images" below.
+  can just regenerate the test image.  See "[Regenerating reference
+  images][#regenerating-reference-images]" below.
 
 * If the `foo-out.png` image is obviously wrong when compared to the
-  `foo-ref.png` reference, you can file a bug in
-  https://bugzilla.gnome.org.  You can wait until someone fixes it, or
-  try to fix the bug yourself!
+  `foo-ref.png` reference, you can [file a bug][bug].  You can wait
+  until someone fixes it, or try to [fix the bug yourself][pull-requests]!
 
-* Any other situation of course deserves attention.  Feel free to ask
-  the maintainers about it; even if you figure out the problem
+* Any other situation of course deserves attention.  Feel free to [ask
+  the maintainers][mail] about it; even if you figure out the problem
   yourself, a failed test almost always indicates a problem that is
-  not just on your end.  See the `librsvg/MAINTAINERS` file for where
-  to reach them.
+  not just on your end.
 
 
 ### Regenerating reference images
 
 Let's continue with the example above, where the test
-`/rsvg-test/reftests/svg1.1/paths-data-18-f.svg` failed.  Let's say you
-fix the bug, or determine that the output image is in fact correct,
-and just differs from the reference image due to antialiasing
-artifacts.  In this case, you'll want to regenerate the reference
-image so the test passes again.
+`/rsvg-test/reftests/svg1.1/paths-data-18-f.svg` failed.  Let's say
+you fix the bug, or determine that the output image is in fact
+correct, and just differs from the reference image due to antialiasing
+artifacts.  In this case, your next step is to regenerate the
+reference image so the test passes again.
 
 If you want to regenerate the reference image
 `fixtures/reftests/foo/bar-ref.png` from the corresponding `bar.svg`, you can run
@@ -188,26 +197,33 @@ suite are pixel-aligned, and some are not, like the example test-frame
 above.  It looks like their renderer uses a (0.5, 0.5) offset just for
 strokes, but not for fills, which sounds hackish.
 
-Our test suite doesn't use special offsets so that SVG images not from
-the official test suite are rendered "normally".  This means that the
-reference images from the official test suite will always fail
-initially, since stroke outlines will be fuzzy in librsvg, but not in
-the test suite (and conversely, SVGs *not* from the test suite would
-be crips in librsvg but probably not in the test suite's renderer
-renderer).
+Our test suite **does not** use special offsets, so that SVG images
+not from the official test suite are rendered "normally".  **This means
+that the reference images from the official test suite will always
+fail initially**, since stroke outlines will be fuzzy in librsvg, but
+not in the test suite (and conversely, SVGs *not* from the test suite
+would be crisp in librsvg but probably not in the test suite's
+renderer renderer).
 
-Also, the original reference PNGs either use fonts that are different
-from those usually on free software systems, or they use SVG fonts
-which librsvg currently doesn't support (with glyph shapes referenced
-from a secondary SVG).
+Also, the original reference PNGs from the SVG 1.1 test suite either
+use fonts that are different from those usually on free software
+systems, or they use SVG fonts which librsvg currently doesn't support
+(i.e. with glyph shapes referenced from a secondary SVG).
 
 In any case, look at the results by hand, and compare them by eye to
 the official reference image.  If the thing being tested looks
 correct, and just the outlines are fuzzy — and also it is just the
 actual font shapes that are different — then the test is probably
-correct.  Follow the procedure as in "Regenerating reference images"
-listed above in order to have a reference image suitable for librsvg.
+correct.  Follow the procedure as in "[Regenerating reference
+images][#regenerating-reference-images]" listed above in order to have
+a reference image suitable for librsvg.
+
 
+## Loading tests for `loading.c`
+
+These test the code that decompresses compressed SVGs and feeds the
+XML reader, by tring to load SVG data one or two bytes at a time.  The
+SVG and SVGZ images are in the `fixtures/loading` directory.
 
 ## Crash tests for `crash.c`
 
@@ -219,9 +235,24 @@ program will look recursively look inside `fixtures/crash` for all SVG
 files, load them, and ensure that no GError was produced while loading
 each file.
 
+## Rendering crash tests for `render-crash.c`
+
+We use these tests to ensure there are no regressions after fixing a
+bug where a particular SVG loads fine, but it crashes the renderer.
+The test files are in the `fixtures/render-crash` directory.
+
+## Dimensions tests for `dimensions.c`
+
+Here we test that librsvg computes the correct dimensions for objects
+in an SVG file.  The test files are in the `fixtures/dimensions`
+directory.  The expected dimensions are declared in the test fixtures
+in `dimensions.c`.
 
 ## Style tests for `styles.c`
 
+These tests are currently disabled, since they depend on internal APIs
+that are not visible to the test suite.
+
 These load and parse an SVG, ask librsvg to fetch an SVG object by its
 id, and ensure that the object has certain style properties with
 particular values.
@@ -229,3 +260,9 @@ particular values.
 The SVG images live in the `fixtures/styles` directory.  The
 corresponding object IDs and values to be tested for are in the
 `styles.c` source code.
+
+
+[gtest]: https://developer.gnome.org/glib/stable/glib-Testing.html
+[bug]: ../CONTRIBUTING.md#reporting-bugs
+[pull-requests]: ../CONTRIBUTING.md#pull-requests
+[mail]: mailto:federico gnome org


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]