Merge branch 'camera-defaults-patch' into 'v3'

Set colour correction to interpolate value

See merge request openflexure/openflexure-microscope-server!414

hardware-specific-tests/README.md

@@ -10,24 +10,12 @@ To test the code that interacts with the hardware itself, hardware specific test
 
 In GitLab we track our "code coverage", this tells us which lines of code have been executed during testing. The `picamera_tests.py` script in the root of the repository can be used to report the coverage from these tests. See the section below on "Reporting the coverage to GitLab".
 
-### Running these tests during development
+### Running and reporting the test coverage to GitLab (for merge requests)
 
-These are very slow as they run on hardware. They can be run with:
-
-    pytest hardware-specific-tests
-
-It is essential to stop the server first:
+It is essential to stop the server before running these tests:
 
     ofm stop
 
-However, this will not archive the tests in the Git repository for reporting the coverage. For this see the section below on reporting the coverage.
-
-When writing and debugging these unit tests it is often best to run a specific test and to use the `-s` flag to see the print statements. It is also often useful to use `--pdb` to drop you into a python debug session on any failure. For example, you might run:
-
-    pytest hardware-specific-tests/picamera2/test_exposure_time_drift.py::test_exposure_time_saves_and_loads -s --pdb
-
-### Reporting the coverage to GitLab
-
 To create a coverage report that will be included into the repository (and reported to GitLab) run:
 
     ./picamera_tests.py run
@@ -40,19 +28,35 @@ This will first run `pytest` on the hardware specific tests. This creates a `.co
 
 This zip should then be committed to the repository.
 
-When the CI runs on GitLab to calculate code coverage. It will first run the tests in the `tests` directory, in one job and archive this as `.coverage.main`, and then it will run a second job that:
+### Running these tests during development
+
+As before, the server must be stopped before running these tests.
+
+The camera test are very slow as they run on hardware. They can be run with:
+
+    pytest hardware-specific-tests
+
+However, this will not archive the tests in the Git repository for reporting the coverage. For this, see the section above on reporting the coverage.
+
+When writing and debugging these unit tests, it is often best to run a specific test and to use the `-s` flag to see the print statements. It is also often useful to use `--pdb` to drop you into a python debug session on any failure. For example, you might run:
+
+    pytest hardware-specific-tests/picamera2/test_exposure_time_drift.py::test_exposure_time_saves_and_loads -s --pdb
+
+### CI explanation
+
+When the CI runs on GitLab to calculate code coverage, it will first run the tests in the `tests` directory in one job, and archive this as `.coverage.main`. Then it will run a second job that:
 
 * Imports the archive of `.coverage.main` for the tests just run on the server
 * Unzip the zip of the results of running tests on the Pi Camera (`.coverage.picamera`)
 * Check that the hashes for the Pi Camera source code have not changed. If they have changed it will error and ask for the picamera tests to be re-run on a Raspberry Pi.
-* It will then run `coverage combine` to create a single `.coverage` report that combines the coverage from both `.coverage.main` and `.coverage.picamera`.
-* It then generates the information needed to display the coverage in GitLab
+* Runs `coverage combine` to create a single `.coverage` report that combines the coverage from both `.coverage.main` and `.coverage.picamera`.
+* Generates the information needed to display the coverage in GitLab
 
 This ensures that:
 
 * Hardware specific tests are re-run if the relevant source code changes.
-* That the coverage for hardware specific tests is reported correctly
-* That the hardware specific tests do not need re-running when other code that does not affect PiCamera interaction is updated.
+* The coverage for hardware specific tests is reported correctly
+* The hardware specific tests do not need re-running when other code that does not affect PiCamera interaction is updated.
 
 ### Creating a combined report locally
 

src/openflexure_microscope_server/things/camera/picamera.py

@@ -772,10 +772,11 @@ class StreamingPiCamera2(BaseCamera):
         This is broken out into its own property for convenience and compatibility with
         the micromanager API
 
-        Ir is a 9 value tuple used to specify the 3x3 matrix that the GPU pipeline uses
+        It is a 9 value tuple used to specify the 3x3 matrix that the GPU pipeline uses
         to convert from the camera R,G,B vector to the standard R,G,B.
 
-        See page Raspberry Pi Camera Algorithm and Tuning Guide, page 45.
+        The value here is interpolated from the IMX219 defaults for the colour temperatures
+        above and below our LED temperature of 5000K.
         """
         return tuple(tf_utils.get_static_ccm(self.tuning)[0]["ccm"])
 
@@ -798,17 +799,31 @@ class StreamingPiCamera2(BaseCamera):
         45.
         """
         # This is flattened 3x3 matrix. See `colour_correction_matrix`
-        col_corr_matrix = [
-            1.80439,
-            -0.73699,
-            -0.06739,
-            -0.36073,
-            1.83327,
-            -0.47255,
-            -0.08378,
-            -0.56403,
-            1.64781,
-        ]
+        if self._camera_board == "picamera_v2":
+            col_corr_matrix = [
+                2.222935,
+                -0.759672,
+                -0.463262,
+                -0.683489,
+                2.711882,
+                -1.028399,
+                -0.261375,
+                -0.668016,
+                1.929391,
+            ]
+        else:
+            # Note the only other option is the HQ
+            col_corr_matrix = [
+                2.164374,
+                -0.97259,
+                -0.191778,
+                -0.376957,
+                2.099377,
+                -0.722417,
+                -0.11787,
+                -0.489362,
+                1.607232,
+            ]
         self.colour_correction_matrix = col_corr_matrix
 
     @lt.thing_action
@@ -858,6 +873,7 @@ class StreamingPiCamera2(BaseCamera):
         self.calibrate_lens_shading()
         self.reset_ccm()
         self.calibrate_white_balance()
+        time.sleep(0.5)
         self.set_background(portal)
 
     @lt.thing_action

src/openflexure_microscope_server/things/camera/picamera_tuning_file_utils.py

@@ -64,7 +64,7 @@ def set_static_ccm(
     adaptive tweaking by the algorithm.
     """
     ccm = Picamera2.find_tuning_algo(tuning, "rpi.ccm")
-    ccm["ccms"] = [{"ct": 2860, "ccm": col_corr_matrix}]
+    ccm["ccms"] = [{"ct": 5000, "ccm": col_corr_matrix}]
 
 
 def get_static_ccm(tuning: dict) -> None: