| |
| The purpose of this autotest is to verify that given subset of PPD files work |
| in ChromeOS. Each PPD file is tested with the following procedure: |
| 1. A printer driver is added to CUPS server. |
| 2. Test documents are printed on the configured printer. |
| 3. Raw output from the CUPS server is intercepted by, so called, FakePrinter. |
| 4. Obtained outputs are verified (see below). |
| 5. The printer driver is removed from CUPS server. |
| This procedure is repeated for every PPD file. The number of PPD files may be |
| large (3K+ files). To decrease amount of time needed by the autotest, several |
| PPD files are tested simultaneously in parallel threads. Autotest always run |
| the procedure for all given PPD files and print a summary report at the end. |
| If at least one of PPD files fails, whole autotest is finished with failure |
| (but always all PPD files are processed). |
| |
| Output verification: |
| Intercepted output is verified by comparision with the previous results |
| obtained for the same PPD. We cannot store outputs directly, because their |
| total size may have hundreds of megabytes. Instead of that short digest is |
| calculated for each obtained document and it is used for comparision. |
| A function for digests calculation is in the 'helpers.py' file. Not all |
| outputs can be tested this way because for some PPD files produced contents |
| differ between runs. List of PPD files for which we cannot calculate |
| constant digest is saved in the file digests_blacklist.txt. Files with |
| expected digests for every test document are stored in the directory "digests". |
| If a digests for given pair (test document, PPD file) is missing, the test |
| checks only check if the output is not empty (or not too short). |
| |
| Parameters: |
| path_docs - path to directory with test documents (PDF files) |
| path_ppds - path to directory with PPD files, if not set then all available PPD |
| files are downloaded and tested |
| path_digests - path to directory with files containing digests for |
| verification, if not set then outputs are not verified |
| path_outputs - if set, then all outputs are dumped there (given directory is |
| deleted if already exists); also all digests files are recalculated |
| and saved in the same directory |
| |
| Generating new digests: |
| The following procedure can be used to update digests: |
| 1. Run the test defined in control.all_outputs: |
| test_that <device IP> PrinterPpds_outputs |
| 2. Download generated files with digests to your workstation |
| rsync root@<device IP>:/tmp/PrinterPpds_outputs/*.digests <local dir> |
| 3. Replace the files from the "digests" directory and commit changes |
| |
| Updating the archive with PPD files: |
| Currently, all tests are based on PPD files stored in local directories. The |
| autotest can download all PPD files by itself, but we do not use this option |
| to limit number of possible points of failures during testing. PPD files are |
| stored in the archive 'ppds_all.tar.xz'. To replace the archive with the current |
| list of supported PPD files one can use the script |
| 'download_ppds_make_archive.py'. |
| |
| Testing printing pipelines outside CUPS server: |
| The test procedure can also include testing of printing pipelines outside CUPS |
| server. To enable this routine, the following conditions must be met: |
| 1. The patch 'cups_log_pipelines.patch' must be applied to CUPS server before |
| the test is started. |
| 2. The test parameter path_outputs must be set. |
| When both these conditions are fulfilled, patched CUPS saved in /tmp directory |
| every executed printing pipeline. At the end of the test procedure described at |
| the beginning of this document, whole pipeline is rerun outside CUPS and every |
| intermediate content is saved to the directory given in path_outputs parameter. |
| Additionally, each pipeline's step is verified by checking its return code (it |
| is supposed to be equal 0) and the final output is compared with the output |
| returned by CUPS server. This mode requires more disk space on tested device |
| (~2GB in /tmp) and takes more execution time (~2 times more) but allows to |
| detect errors that are often ignored silently by CUPS server. |
| |
| Others: |
| * How to get comma-separated list of all used ghostscript devices? |
| 1. Go to the directory generated by all_outputs (with patched CUPS!) |
| 2. Unpack all tar.xz archives (not on the device) |
| for d in *.pdf; |
| do |
| cd $d; |
| for f in *.tar.xz; do tar xf $f; done ; |
| cd ..; |
| done |
| 3. Run the following |
| grep -o sDEVICE=[^\ ]*\ ./*.pdf/*.err[12345] | cut -d \= -f 2 | sort | uniq | tr -d ' ' | tr '\n' ',' |
| 4. Add to the list uniprint (it is not caught by the command above) |