Merge branch 'more-dev-docs' into 'v3'
More developer documnetation See merge request openflexure/openflexure-microscope-server!513
This commit is contained in:
Julian Stirling
commit
e443b1ac5b
17 changed files with 3583 additions and 225 deletions
|
|
@ -7,7 +7,10 @@ skip = node_modules,
|
|||
__pycache__,
|
||||
./build,
|
||||
./dist,
|
||||
./apidocs,
|
||||
./apidocs/*.html,
|
||||
./apidocs/*.js,
|
||||
./apidocs/*.json,
|
||||
./apidocs/*.css,
|
||||
./openflexure,
|
||||
./htmlcov,
|
||||
./src/openflexure_microscope_server/static,
|
||||
|
|
|
|||
|
|
@ -176,7 +176,10 @@ build:
|
|||
|
||||
# Only runs when a .py file is edited unless the commit branch is V3
|
||||
build-docs:
|
||||
extends: .python
|
||||
extends:
|
||||
- .python
|
||||
# Always build docs
|
||||
- .rules_common
|
||||
stage: build
|
||||
script:
|
||||
- |
|
||||
|
|
@ -200,17 +203,7 @@ build-docs:
|
|||
environment:
|
||||
name: review/$CI_COMMIT_REF_SLUG
|
||||
url: https://$CI_PROJECT_ROOT_NAMESPACE.gitlab.io/-/openflexure-microscope-server/-/jobs/$JOB_ID/artifacts/apidocs/index.html
|
||||
rules:
|
||||
# So that 'pages' job at the end always works
|
||||
# Add new rule first as the rules defined in .python
|
||||
# end in when: never, so if this new rule goes after that
|
||||
# clause, it will never be reached as rules are
|
||||
# evaluated top to bottom
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
# This syntax means we append the rules from the .python
|
||||
# job to this job, instead of overwriting them by creating
|
||||
# this new rules section
|
||||
- !reference [.python, rules]
|
||||
|
||||
|
||||
# Only runs when a .py file is edited
|
||||
server_lifecycle_test:
|
||||
|
|
|
|||
19
README.md
19
README.md
|
|
@ -168,22 +168,3 @@ We use several code analysis and formatting libraries in this project. Our CI wi
|
|||
### Python environment, build, and dependencies
|
||||
|
||||
As of `v3` we specify dependencies in `pyproject.toml`. These are not currently frozen due to difficulties matching versions on different platforms. On Raspberry Pi, it's best to specify `--only-binary=:all:` to ensure libraries like `numpy` and `scipy` are not compiled from source (which takes many hours, and/or fails). Pinning dependencies with `requirements.txt` and/or `requirements.in` may happen in the future.
|
||||
|
||||
|
||||
# Notes for Maintainers
|
||||
|
||||
## Creating releases
|
||||
|
||||
* Update the application's internal version number
|
||||
* Edit `pyproject.toml` to update the version number
|
||||
* Update the changelog
|
||||
* Git commit and git push
|
||||
* Create a new version tag on GitLab (e.g. `v2.6.11`) that matches the `pyproject.toml` version number.
|
||||
* Make sure you prefix a lower case 'v', otherwise it won't be recognised as a release!
|
||||
* This tagging will trigger a CI pipeline that builds the JS client, tarballs up the server, and deploys it
|
||||
* Note: This also updates the build server's nginx redirect map file
|
||||
|
||||
## Changelog generation
|
||||
|
||||
* `npm install -g conventional-changelog-cli`
|
||||
* `npx conventional-changelog -r 1 --config ./changelog.config.js -i CHANGELOG.md -s`
|
||||
|
|
|
|||
9
apidocs/dev/README.md
Normal file
9
apidocs/dev/README.md
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
# Developer Guidance
|
||||
|
||||
This site provides information that is useful for developing the OpenFlexure Microscope.
|
||||
|
||||
### Viewing and Modifying Locally
|
||||
|
||||
If you want to view these documents in your machine (very useful when editing them). You will need to serve the `apidocs` directory. Navigate to the `apidocs` directory and run `python -m http.server`.
|
||||
|
||||
All developer guidance documentation is written in markdown and then displayed with Docsify. More information on Docsify can be found at https://docsify.js.org.
|
||||
9
apidocs/dev/_sidebar.md
Normal file
9
apidocs/dev/_sidebar.md
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
<p><a href="../index.html" class="ext">Developer Documentation Index</a><p>
|
||||
<p><a href="../python/index.html" class="ext">Python API</a><p>
|
||||
<p><a href="../http/index.html" class="ext">HTTP API</a><p>
|
||||
|
||||
<h2>Developer Guidance<h2>
|
||||
|
||||
- [Home](README)
|
||||
- [Software Architecture](architecture.md)
|
||||
- [PiCamera Tuning Files](picamera-tuning-files.md)
|
||||
11
apidocs/dev/architecture.md
Normal file
11
apidocs/dev/architecture.md
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
# Software Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The OpenFlexure Microscope Server is a FastAPI server created using Labthings FastAPI. The user interfaces is a Vue.js web app. And thehe server runs on a customised operating system (Raspbian OpenFlexure).
|
||||
|
||||
Here is a high level diagaram of the overall software architecture:
|
||||
|
||||
<object style="width:100%" type="image/svg+xml" data="./assets/SoftwareDiagram.svg"></object>
|
||||
|
||||
Each repository link should be clickable. The <a href="./assets/SoftwareDiagram.svg" target="_blank">architecture diagram is best viewed in its own tab</a>.
|
||||
BIN
apidocs/dev/assets/ChiefRayAngle.png
Normal file
BIN
apidocs/dev/assets/ChiefRayAngle.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 95 KiB |
BIN
apidocs/dev/assets/GreenEqualisation.png
Normal file
BIN
apidocs/dev/assets/GreenEqualisation.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 109 KiB |
2291
apidocs/dev/assets/SoftwareDiagram.svg
Normal file
2291
apidocs/dev/assets/SoftwareDiagram.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 149 KiB |
43
apidocs/dev/index.html
Normal file
43
apidocs/dev/index.html
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport"
|
||||
content="width=device-width, initial-scale=1, minimum-scale=1.0, shrink-to-fit=no, viewport-fit=cover">
|
||||
|
||||
<title>OpenFlexure Microscope Server Developer Guidance</title>
|
||||
<meta name="description" content="Developer information for the OpenFlexure Microscope Server.">
|
||||
|
||||
<link rel="stylesheet" href="./ofm.css">
|
||||
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
|
||||
<script>
|
||||
// Docsify Configuration (see https://docsify.js.org/#/configuration)
|
||||
window.$docsify = {
|
||||
name: 'Developer Documentation',
|
||||
// Sidebar Configuration
|
||||
auto2top: true,
|
||||
loadSidebar: true,
|
||||
maxLevel: 0,
|
||||
subMaxLevel: 4,
|
||||
|
||||
// Search Plugin Configuration
|
||||
search: {
|
||||
placeholder: 'Type to search',
|
||||
noData: 'No matches found.',
|
||||
depth: 2,
|
||||
}
|
||||
};
|
||||
</script>
|
||||
|
||||
<script src="https://cdn.jsdelivr.net/npm/docsify@4/lib/docsify.min.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/docsify@4/lib/plugins/zoom-image.min.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/docsify@4/lib/plugins/search.js"></script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
1002
apidocs/dev/ofm.css
Normal file
1002
apidocs/dev/ofm.css
Normal file
File diff suppressed because it is too large
Load diff
188
apidocs/dev/picamera-tuning-files.md
Normal file
188
apidocs/dev/picamera-tuning-files.md
Normal file
|
|
@ -0,0 +1,188 @@
|
|||
# Raspberry Pi Camera Tuning Files
|
||||
|
||||
The OpenFlexure Microscope ships its own custom Raspberry Pi Camera tuning files. This page provides more information on these files.
|
||||
|
||||
|
||||
## A list of all the settings in the Raspberry Pi Camera (vc4) tuning files.
|
||||
|
||||
Note that VC4 is the patform used by the Pi for with the broadcom ISP. From Pi 5 onwards Raspberry Pi uses their own ISP (PiSP). These have their own tuning file structure.
|
||||
|
||||
More data is available in this datasheet: https://datasheets.raspberrypi.com/camera/raspberry-pi-camera-guide.pdf
|
||||
|
||||
### version:
|
||||
|
||||
This needs to be 2.0. Version one was a normal dictionary of algorithms, rather than this current structure with the algorithms as a list.
|
||||
|
||||
### target
|
||||
|
||||
Needs to be bcm2835. This is the broadcom system on a chip used by vc4.
|
||||
|
||||
### Algorithms
|
||||
|
||||
#### rpi.black_level - Sensor Black Level
|
||||
|
||||
Contains one key-value pair. i.e.: `black_level: 4096`
|
||||
|
||||
Note that this is not the actual black level of the sensor, but the black level once the image has been scaled up to 16 bit. So for a 10-bit sensor this number would need to be divided by 2^6.
|
||||
|
||||
#### rpi.dpc - Defective Pixel Correction
|
||||
|
||||
This has three strength parameters:
|
||||
|
||||
* 0 - None
|
||||
* 1 - Normal (default)
|
||||
* 2 - Strong
|
||||
|
||||
#### rpi.lux - Lux Calculation
|
||||
|
||||
This is used to create the lux value. This lux value is not used in the ISP but is sent to the metadata for the image for downstream control algorithms. The camera may crash if this is removed even though we turn off the adaptive algorithms.
|
||||
|
||||
#### rpi.noise - Noise Profile
|
||||
|
||||
This calculates the noise profile for the current images.
|
||||
|
||||
Values from the default tuning files for imx219 and imx477 are:
|
||||
|
||||
```
|
||||
"reference_constant": 0,
|
||||
"reference_slope": 3.67
|
||||
```
|
||||
|
||||
These can be optimised using the methods in the camera tuning tool shipped with libcamera. Though most of the camera tuning tool methods make assumptions that are invalid for a microscope. As such any tuning would need further consideration.
|
||||
|
||||
#### rpi.geq - Green Equalisation
|
||||
|
||||
##### The problems with green equalisation
|
||||
|
||||
This tried to calculate whether adjacent grren pixels should be equal. This interacts very porly with the chief ray angle compensation when using the camera. The imx219 has a lenslet array:
|
||||
|
||||
|
||||
|
||||
Normally as shown in (a) the light comes in at an angle as we get to the edge of the sensor due to how close the lens is to the camera. The lenslet array is designed for this and so if you are in the centre (b), or at the edge (d) the light is focussed onto the correct pixel.
|
||||
|
||||
For a microscope the light comes in perpendicular to the sensor, this makes no difference in the centre but at the edge of the image (e) there is a lot of cross talk between pixels.
|
||||
|
||||
But it makes sense that this also strongly affects the green imbalance (which is something I didn't know about, but should, and will need to read about in more detail so we can improve tuning). If we consider the case of red light, we expect that the red pixels are the only illuminated pixels. If we consider what happens on the sensor in the case of the microscope, I think the light will be focussing on the positions shown:
|
||||
|
||||
|
||||
|
||||
So in the case of the diagonals this is illuminating the green in both the red and the blue row. Where as in the areas we were seeing issues the red light will be strongly illuminating the the greens in one row but not the other.
|
||||
|
||||
The result is that for strongly red on the edges every other geen pixel is illuminated causing an overflow.
|
||||
|
||||
##### Turning off the adaptive green equalisation
|
||||
|
||||
The parameters for geeen equalisation are a `slope` and `offset`
|
||||
|
||||
The "green equalisation" algorithm averages Gr and Gb when they "don't look that different". The threshold below which things get averaged is given by
|
||||
|
||||
`threshold = offset + pixel_value * slope`
|
||||
|
||||
As we always want the green pixels to be averaged to remove the effect above. We can set `offset` the maximum 16-bit value (65535). In this case it will always average green pixels, no matter the pixel value or the value of `slope`.
|
||||
|
||||
#### rpi.sdn - Spatial Denoise
|
||||
|
||||
This algorithm is used on the VC4 (Pi 4) but not on PiSP (Pi 5) which uses `rpi.denoise`. A warning on startup asks us to move it into `rpi.denoise` even though we are on VC4 that does not support `rpi.denoise`.
|
||||
|
||||
#### rpi.awb - Automatic White Balance
|
||||
|
||||
Even though we don't use it, camera can't start without it being minimally populated.
|
||||
|
||||
#### rpi.agc - Automatic Gain Control / Automatic Exposure Control
|
||||
|
||||
For calculating the shutter time of the sensor if AeEnable is on. Only the required fields are populated with minimal data as AeEnable is always off.
|
||||
|
||||
#### rpi.alsc - Automatic Lens Shading Correction
|
||||
|
||||
Auto lens shading correction is calibrated by our microscope. This contains the tables for luninance, and colour shading. It can contain multiple tables depending on estimated colour temperature (`ct`). As the microscope has fixed lighting we only provide one set of tables as we don't want the correction to be adaptive.
|
||||
|
||||
We also set `n_iter: 0`. This stops it adding iterative adaptuion.
|
||||
|
||||
As there is only one table, the colour temperature is ignored. We set the colour temperature to 1234 in the tuning file to make it clear that we have not calibrated it on the microscope. We set the colour temperature to 5000 (our actual illumination colour temperature) once recalibrated.
|
||||
|
||||
The tables are flat 1s for luminance on for both sensors.
|
||||
|
||||
For the IMX477 (PiCamera HQ), the Cr and Cb tables are flat, but Cb is 2.0 not 1.0 for all values. This is so that the initial colour gains are set to (1.0, 2.0)
|
||||
|
||||
For the IMX219 (PiCamera v2), the Cr and Cb tables are symmetric tables taken from the top quadrant of a calibrated microscope. They are used to create a good enough correction that the microscope is initially usable for alignment.
|
||||
|
||||
#### rpi.contrast - Contrast
|
||||
|
||||
This does the gamma correction. If `ce_enable` is 1 then the gamma correction is somewhat adaptive. We set this to zero to always use the specified gamma table.
|
||||
|
||||
The table of gamma values is a 1d list of alternating x,y values in the gamma curve.
|
||||
|
||||
We may want to adjust the gamma curve in future.
|
||||
|
||||
#### rpi.ccm - Colour Correction Matrix
|
||||
|
||||
It is possible tp ship different CCMs to be used based on the estimated colour temperatire. We only ship one CCM as our illumination colour temperature is fixed to 5000K, the colour temperature of the LED board. However, the diffuser might affect this.
|
||||
|
||||
The CCMs are different for PiCamera HQ and PiCamera version 2. The default tuning files for these cameras (`/usr/share/libcamera/ipa/rpi/vc4/imx219.json` or `/usr/share/libcamera/ipa/rpi/vc4/imx477.json`) specify CCMs for a number of temperatures but not for 5000K. As such we linearly interpolate between temperatures around 5000K with the following code:
|
||||
|
||||
```
|
||||
import numpy as np
|
||||
|
||||
temps = {4650:
|
||||
np.array([
|
||||
2.18174, -0.70887, -0.47287,
|
||||
-0.70196, 2.76426, -1.06231,
|
||||
-0.25157, -0.71978, 1.97135
|
||||
]),
|
||||
5858:
|
||||
np.array(
|
||||
[
|
||||
2.32392, -0.88421, -0.43971,
|
||||
-0.63821, 2.58348, -0.94527,
|
||||
-0.28541, -0.54112, 1.82653
|
||||
])
|
||||
}
|
||||
target_temp = 5000
|
||||
temp_low, temp_high = sorted(temps.keys())
|
||||
alpha = (target_temp - temp_low) / (temp_high - temp_low)
|
||||
|
||||
print(alpha)
|
||||
|
||||
ccm_interp = (1 - alpha) * temps[temp_low] + alpha * temps[temp_high]
|
||||
print(np.round(ccm_interp, 6))
|
||||
|
||||
temps = {4559: np.array(
|
||||
[
|
||||
2.15423, -0.98143, -0.17279,
|
||||
-0.38131, 2.14763, -0.76632,
|
||||
-0.10069, -0.54383, 1.64452
|
||||
]),
|
||||
5881:np.array(
|
||||
[
|
||||
2.18464, -0.95493, -0.22971,
|
||||
-0.36826, 2.00298, -0.63471,
|
||||
-0.15219, -0.38055, 1.53274
|
||||
])
|
||||
}
|
||||
target_temp = 5000
|
||||
temp_low, temp_high = sorted(temps.keys())
|
||||
alpha = (target_temp - temp_low) / (temp_high - temp_low)
|
||||
|
||||
print(alpha)
|
||||
|
||||
ccm_interp = (1 - alpha) * temps[temp_low] + alpha * temps[temp_high]
|
||||
print(np.round(ccm_interp, 6))
|
||||
```
|
||||
|
||||
##### Calculating our own CCMs
|
||||
|
||||
The CCM calculation is done using a MacBeth target. Lib camera provides a method for this using the Colour Tuning Tool. We will need to adapt this algorithm as they image the target at once and pick out the areas. We have fixed lighting but do not have colour uniformity.
|
||||
|
||||
In the future, with a colour calibration slide, we can take images of each colour with fixed camera settings to create a CCM using the same calculations as the Camera Tuning Tool.
|
||||
|
||||
#### rpi.sharpen - Sharpening
|
||||
|
||||
This is empty. We do not want to be artificially sharpening images as this can add in artifacts.
|
||||
|
||||
#### rpi.hdr - High dynamic range
|
||||
|
||||
Not relevant for us, we don't use high dynamic range capture.
|
||||
|
||||
#### rpi.sync - Software Camera Synchronisation
|
||||
|
||||
Not relevant for us, we only have 1 camera.
|
||||
|
|
@ -64,6 +64,10 @@
|
|||
<code>http://microscope.local:5000/docs</code>
|
||||
</p>
|
||||
|
||||
<p>
|
||||
You can also directly inspect the OpenAPI JSON file <a href="./openapi.json">openapi.json</a> or the WC3 "Web of Things" Thing Descriptions JSON <a href="./thing_descriptions.json">thing_descriptions.json</a>
|
||||
</p>
|
||||
|
||||
<aside>
|
||||
Replace <strong>microscope.local</strong> with your microscope's hostname or IP address
|
||||
if your network uses a different name.
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>OpenFlexure Microscope Server API Docs</title>
|
||||
<title>OpenFlexure Microscope Server Developer Documentation</title>
|
||||
|
||||
<link href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@400;600&display=swap" rel="stylesheet">
|
||||
|
||||
|
|
@ -21,9 +21,15 @@ body {
|
|||
padding: 0 1.5rem;
|
||||
}
|
||||
|
||||
.main-content {
|
||||
max-width: 700px;
|
||||
padding: 2rem;
|
||||
}
|
||||
|
||||
h1 {
|
||||
color: #c5247f;
|
||||
margin-bottom: 0.5rem;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
a {
|
||||
|
|
@ -36,6 +42,7 @@ a {
|
|||
font-weight: 600;
|
||||
border-radius: 6px;
|
||||
width: fit-content;
|
||||
min-width: 15rem;
|
||||
transition: 0.15s ease;
|
||||
}
|
||||
|
||||
|
|
@ -49,11 +56,14 @@ a:hover {
|
|||
<body>
|
||||
|
||||
<div class="content">
|
||||
<h1>OpenFlexure Microscope Server API Docs</h1>
|
||||
<p>Select documentation:</p>
|
||||
<h1>OpenFlexure Microscope Server Developer Documentation</h1>
|
||||
<div class="main-content">
|
||||
<p>Select documentation:</p>
|
||||
|
||||
<a href="./http/index.html">HTTP API (Swagger)</a>
|
||||
<a href="./python/index.html">Python API</a>
|
||||
<a href="./http/index.html">HTTP API (Swagger)</a>
|
||||
<a href="./python/index.html">Python API</a>
|
||||
<a href="./dev/index.html">Developer Guidance</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</body>
|
||||
|
|
|
|||
|
|
@ -1,3 +0,0 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:36ca9e6e8ee0af71a03dee4a6e2d4405c63d54ea9b123046d39c0187402504da
|
||||
size 96857
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8f65ef3712cb9986cb28f40000ddb283e89fe93671b55ad12d8b6fc057afeac9
|
||||
size 111804
|
||||
|
|
@ -1,183 +1,3 @@
|
|||
# A list of all the settings in the Raspberry Pi Camera (vc4) tuning files.
|
||||
## Tuning files.
|
||||
|
||||
Note that VC4 is the patform used by the Pi for with the broadcom ISP. From Pi 5 onwards Raspberry Pi uses their own ISP (PiSP). These have their own tuning file structure.
|
||||
|
||||
More data is available in this datasheet: https://datasheets.raspberrypi.com/camera/raspberry-pi-camera-guide.pdf
|
||||
|
||||
## version:
|
||||
|
||||
This needs to be 2.0, version one was a normal dictionary of algorithms, rather than this current structure with the algorithms as a list.
|
||||
|
||||
## target
|
||||
|
||||
Needs to be bcm2835. This is the broadcom system on a chip used by vc4.
|
||||
|
||||
## Algorithms
|
||||
|
||||
### rpi.black_level - Sensor Black Level
|
||||
|
||||
Contains one key-value pair. i.e.: `black_level: 4096`
|
||||
|
||||
Note that this is not the actual black level of the sensor, but the black level once the image has been scaled up to 16 bit. So for a 10-bit sensor this number would need to be divided by 2^6.
|
||||
|
||||
### rpi.dpc - Defective Pixel Correction
|
||||
|
||||
This has three strength parameters:
|
||||
|
||||
* 0 - None
|
||||
* 1 - Normal (default)
|
||||
* 2 - Strong
|
||||
|
||||
### rpi.lux - Lux Calculation
|
||||
|
||||
This is used to create the lux value. This lux value is not used in the ISP but is sent to the metadata for the image for downstream control algorithms. The camera may crash if this is removed even though we turn off the adaptive algorithms.
|
||||
|
||||
### rpi.noise - Noise Profile
|
||||
|
||||
This calculates the noise profile for the current images.
|
||||
|
||||
Values from the default tuning files for imx219 and imx477 are:
|
||||
|
||||
```
|
||||
"reference_constant": 0,
|
||||
"reference_slope": 3.67
|
||||
```
|
||||
|
||||
These can be optimised using the methods in the camera tuning tool shipped with libcamera. Though most of the camera tuning tool methods make assumptions that are invalid for a microscope. As such any tuning would need further consideration.
|
||||
|
||||
### rpi.geq - Green Equalisation
|
||||
|
||||
#### The problems with green equalisation
|
||||
|
||||
This tried to calculate whether adjacent grren pixels should be equal. This interacts very porly with the chief ray angle compensation when using the camera. The imx219 has a lenslet array:
|
||||
|
||||
|
||||
|
||||
Normally as shown in (a) the light comes in at an angle as we get to the edge of the sensor due to how close the lens is to the camera. The lenslet array is designed for this and so if you are in the centre (b), or at the edge (d) the light is focussed onto the correct pixel.
|
||||
|
||||
For a microscope the light comes in perpendicular to the sensor, this makes no difference in the centre but at the edge of the image (e) there is a lot of cross talk between pixels.
|
||||
|
||||
But it makes sense that this also strongly affects the green imbalance (which is something I didn't know about, but should, and will need to read about in more detail so we can improve tuning). If we consider the case of red light, we expect that the red pixels are the only illuminated pixels. If we consider what happens on the sensor in the case of the microscope, I think the light will be focussing on the positions shown:
|
||||
|
||||
|
||||
|
||||
So in the case of the diagonals this is illuminating the green in both the red and the blue row. Where as in the areas we were seeing issues the red light will be strongly illuminating the the greens in one row but not the other.
|
||||
|
||||
The result is that for strongly red on the edges every other geen pixel is illuminated causing an overflow.
|
||||
|
||||
#### Turning off the adaptive green equalisation
|
||||
|
||||
The parameters for geeen equalisation are a `slope` and `offset`
|
||||
|
||||
The "green equalisation" algorithm averages Gr and Gb when they "don't look that different". The threshold below which things get averaged is given by
|
||||
|
||||
`threshold = offset + pixel_value * slope`
|
||||
|
||||
As we always want the green pixels to be averaged to remove the effect above. We can set `offset` the maximum 16-bit value (65535). In this case it will always average green pixels, no matter the pixel value or the value of `slope`.
|
||||
|
||||
### rpi.sdn - Spatial Denoise
|
||||
|
||||
This algorithm is used on the VC4 (Pi 4) but not on PiSP (Pi 5) which uses `rpi.denoise`. A warning on startup asks us to move it into `rpi.denoise` even though we are on VC4 that does not support `rpi.denoise`.
|
||||
|
||||
### rpi.awb - Automatic White Balance
|
||||
|
||||
Even though we don't use it, camera can't start without it being minimally populated.
|
||||
|
||||
### rpi.agc - Automatic Gain Control / Automatic Exposure Control
|
||||
|
||||
For calculating the shutter time of the sensor if AeEnable is on. Only the required fields are populated with minimal data as AeEnable is always off.
|
||||
|
||||
### rpi.alsc - Automatic Lens Shading Correction
|
||||
|
||||
Auto lens shading correction is calibrated by our microscope. This contains the tables for luninance, and colour shading. It can contain multiple tables depending on estimated colour temperature (`ct`). As the microscope has fixed lighting we only provide one set of tables as we don't want the correction to be adaptive.
|
||||
|
||||
We also set `n_iter: 0`. This stops it adding iterative adaptuion.
|
||||
|
||||
As there is only one table, the colour temperature is ignored. We set the colour temperature to 1234 in the tuning file to make it clear that we have not calibrated it on the microscope. We set the colour temperature to 5000 (our actual illumination colour temperature) once recalibrated.
|
||||
|
||||
The tables are flat 1s for luminance on for both sensors.
|
||||
|
||||
For the IMX477 (PiCamera HQ), the Cr and Cb tables are flat, but Cb is 2.0 not 1.0 for all values. This is so that the initial colour gains are set to (1.0, 2.0)
|
||||
|
||||
For the IMX219 (PiCamera v2), the Cr and Cb tables are symmetric tables taken from the top quadrant of a calibrated microscope. They are used to create a good enough correction that the microscope is initially usable for alignment.
|
||||
|
||||
### rpi.contrast - Contrast
|
||||
|
||||
This does the gamma correction. If `ce_enable` is 1 then the gamma correction is somewhat adaptive. We set this to zero to always use the specified gamma table.
|
||||
|
||||
The table of gamma values is a 1d list of alternating x,y values in the gamma curve.
|
||||
|
||||
We may want to adjust the gamma curve in future.
|
||||
|
||||
### rpi.ccm - Colour Correction Matrix
|
||||
|
||||
It is possible tp ship different CCMs to be used based on the estimated colour temperatire. We only ship one CCM as our illumination colour temperature is fixed to 5000K, the colour temperature of the LED board. However, the diffuser might affect this.
|
||||
|
||||
The CCMs are different for PiCamera HQ and PiCamera version 2. The default tuning files for these cameras (`/usr/share/libcamera/ipa/rpi/vc4/imx219.json` or `/usr/share/libcamera/ipa/rpi/vc4/imx477.json`) specify CCMs for a number of temperatures but not for 5000K. As such we linearly interpolate between temperatures around 5000K with the following code:
|
||||
|
||||
```
|
||||
import numpy as np
|
||||
|
||||
temps = {4650:
|
||||
np.array([
|
||||
2.18174, -0.70887, -0.47287,
|
||||
-0.70196, 2.76426, -1.06231,
|
||||
-0.25157, -0.71978, 1.97135
|
||||
]),
|
||||
5858:
|
||||
np.array(
|
||||
[
|
||||
2.32392, -0.88421, -0.43971,
|
||||
-0.63821, 2.58348, -0.94527,
|
||||
-0.28541, -0.54112, 1.82653
|
||||
])
|
||||
}
|
||||
target_temp = 5000
|
||||
temp_low, temp_high = sorted(temps.keys())
|
||||
alpha = (target_temp - temp_low) / (temp_high - temp_low)
|
||||
|
||||
print(alpha)
|
||||
|
||||
ccm_interp = (1 - alpha) * temps[temp_low] + alpha * temps[temp_high]
|
||||
print(np.round(ccm_interp, 6))
|
||||
|
||||
temps = {4559: np.array(
|
||||
[
|
||||
2.15423, -0.98143, -0.17279,
|
||||
-0.38131, 2.14763, -0.76632,
|
||||
-0.10069, -0.54383, 1.64452
|
||||
]),
|
||||
5881:np.array(
|
||||
[
|
||||
2.18464, -0.95493, -0.22971,
|
||||
-0.36826, 2.00298, -0.63471,
|
||||
-0.15219, -0.38055, 1.53274
|
||||
])
|
||||
}
|
||||
target_temp = 5000
|
||||
temp_low, temp_high = sorted(temps.keys())
|
||||
alpha = (target_temp - temp_low) / (temp_high - temp_low)
|
||||
|
||||
print(alpha)
|
||||
|
||||
ccm_interp = (1 - alpha) * temps[temp_low] + alpha * temps[temp_high]
|
||||
print(np.round(ccm_interp, 6))
|
||||
```
|
||||
|
||||
#### Calculating our own CCMs
|
||||
|
||||
The CCM calculation is done using a MacBeth target. Lib camera provides a method for this using the Colour Tuning Tool. We will need to adapt this algorithm as they image the target at once and pick out the areas. We have fixed lighting but do not have colour uniformity.
|
||||
|
||||
In the future, with a colour calibration slide, we can take images of each colour with fixed camera settings to create a CCM using the same calculations as the Camera Tuning Tool.
|
||||
|
||||
### rpi.sharpen - Sharpening
|
||||
|
||||
This is empty. We do not want to be artificially sharpening images as this can add in artifacts.
|
||||
|
||||
### rpi.hdr - High dynamic range
|
||||
|
||||
Not relevant for us, we don't use high dynamic range capture.
|
||||
|
||||
### rpi.sync - Software Camera Synchronisation
|
||||
|
||||
Not relevant for us, we only have 1 camera.
|
||||
More information on the tuning files can be found in the Developer Guidance. This can be found in the `apidocs/dev` directory, and or hosted on GitLab pages.
|
||||
Loading…
Add table
Add a link
Reference in a new issue