rahix/openflexure-microscope-server
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'better-docstrings' into 'v3'

Browse source 
Better docstrings

Closes #479 and #239

See merge request openflexure/openflexure-microscope-server!320
This commit is contained in:
Julian Stirling 2025-07-11 11:16:27 +00:00
commit 31ed7dfc69
55 changed files with 1365 additions and 1005 deletions
Download patch file Download diff file Expand all files Collapse all files

18
.codespellrc Normal file
View file

@ -0,0 +1,18 @@
[codespell]
skip = node_modules,
*-lock.json,
*.git,
*.pyc,
__pycache__,
./build,
./dist,
./apidocs,
./openflexure,
./htmlcov,
./src/openflexure_microscope_server/static,
.venv,
*.egg-info,
report.xml
regex = [a-zA-Z0-9\-'’]+

4
.gitignore vendored
View file

@ -58,8 +58,8 @@ pytest_report.xml
# Django stuff: # Django stuff:
*.log *.log
# Sphinx documentation # Documentation
docs/_build/ apidocs
# PyBuilder # PyBuilder
target/ target/

46
.gitlab-ci.yml
View file

@ -72,6 +72,13 @@ version-check:
- tags - tags
- web - web
# A job to protect reviewers from Julian's terrible spelling.
spelling:
stage: analysis
extends: .python
script:
- codespell .
# Python static analysis with ruff # Python static analysis with ruff
ruff-lint: ruff-lint:
stage: analysis stage: analysis
@ -168,6 +175,31 @@ build:
paths: paths:
- "src/openflexure_microscope_server/static/" - "src/openflexure_microscope_server/static/"
build-docs:
extends: .python
stage: build
script:
- |
pydoctor -W \
--docformat restructuredtext \
--project-name "OpenFlexure Microscope Server" \
--project-version $CI_COMMIT_REF_NAME \
--theme readthedocs \
--html-viewsource-base https://gitlab.com/openflexure/openflexure-microscope-server/-/tree/$CI_COMMIT_SHA/ \
src/openflexure_microscope_server
- echo "JOB_ID=$CI_JOB_ID" > build-docs.env
artifacts:
expire_in: 1 week
name: "${CI_PROJECT_NAME}-${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}-docs"
paths:
- apidocs
reports:
dotenv: build-docs.env
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://$CI_PROJECT_ROOT_NAMESPACE.gitlab.io/-/openflexure-microscope-server/-/jobs/$JOB_ID/artifacts/apidocs/index.html
server_integration_tests: server_integration_tests:
extends: .python extends: .python
stage: integration stage: integration
@ -261,3 +293,17 @@ server_integration_tests:
# only: # only:
# - tags # - tags
# - web # - web
pages:
needs:
- job: build-docs
artifacts: true
stage: deploy
image: "alpine:latest"
script:
- mv apidocs public
artifacts:
paths:
- public
only:
- v3

2
CHANGELOG.md
View file

@ -83,7 +83,7 @@ The following merge requests have been merged into v3:
* !286 Use same version number for server and webapp, add CI check * !286 Use same version number for server and webapp, add CI check
* !285 Swapped the order of CSM so y axis is completed first * !285 Swapped the order of CSM so y axis is completed first
* !283 Add scan directory to json config * !283 Add scan directory to json config
* !287 Update depedencies and changelog prior to v3.0.0-alpha1 release * !287 Update depedencies and changelog prior to v3.0.0-alpha1 release <!-- codespell:ignore depedencies -->
# [v2.11.0](https://gitlab.com/openflexure/openflexure-microscope-server/compare/v2.10.1...v2.11.0) (2022-08-08) # [v2.11.0](https://gitlab.com/openflexure/openflexure-microscope-server/compare/v2.10.1...v2.11.0) (2022-08-08)
## New features ## New features

2
README.md
View file

@ -2,7 +2,7 @@
> ## This is the v3 development branch > ## This is the v3 development branch
> We are no longer actively developing v2, but v3 is not yet released. > We are no longer actively developing v2, but v3 is not yet released.
> v3 is not yet stable enough for wider public release. Deveopment snapshots are released from time to time. > v3 is not yet stable enough for wider public release. Development snapshots are released from time to time.
> If you want to use a development snapshot before we do a wider alpha release the best thing to do is get in contact on the [forum](https://openflexure.discourse.group/). > If you want to use a development snapshot before we do a wider alpha release the best thing to do is get in contact on the [forum](https://openflexure.discourse.group/).
If you want to look at the code for v2 you should look at the [master branch](https://gitlab.com/openflexure/openflexure-microscope-server/-/tree/master). If you want to look at the code for v2 you should look at the [master branch](https://gitlab.com/openflexure/openflexure-microscope-server/-/tree/master).

6
check_version_sync.py
View file

@ -1,9 +1,5 @@
#! /usr/bin/env python3 #! /usr/bin/env python3
"""Check that the npm version string matches the Python version string exactly."""
"""
A script to check that the npm version string matches the python
version string exactly.
"""
import json import json
import tomllib import tomllib

23
hardware-specific-tests/picamera2/test_acquisition.py
View file

@ -1,3 +1,5 @@
"""Test data collection from the Raspberry Picamera."""
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
from PIL import Image from PIL import Image
import numpy as np import numpy as np
@ -11,10 +13,13 @@ from openflexure_microscope_server.things.camera.picamera import StreamingPiCame
@fixture(scope="module") @fixture(scope="module")
def client(): def client():
""" """Initialise a test client for the StreamingPiCamera2 Thing.
A pytest fixture that initialises a test client for the StreamingPiCamera2 Thing.
This fixture sets up a ThingServer, registers a StreamingPiCamera2 instance at the This fixture:
"/camera/" endpoint, and provides a ThingClient for interacting with it during tests.
* Sets up a ThingServer,
* Registers a StreamingPiCamera2 instance at the "/camera/" endpoint
* Provides a ThingClient for interacting with it during tests.
""" """
server = ThingServer() server = ThingServer()
server.add_thing(StreamingPiCamera2(), "/camera/") server.add_thing(StreamingPiCamera2(), "/camera/")
@ -24,16 +29,14 @@ def client():
def test_calibration(client): def test_calibration(client):
""" """Check that full auto calibrate completes without an exception."""
Check that full auto calibrate completes without an exception
"""
client.full_auto_calibrate() client.full_auto_calibrate()
def test_jpeg_and_array(client): def test_jpeg_and_array(client):
""" """Check that a jpeg grabbed from the stream is the same size as other captures.
Check that grabbing a jpeg from the stream results in the same size
image as a array capture or a jpeg capture. Compare it to an array capture and a jpeg capture.
""" """
# Grab a jpeg from the stream # Grab a jpeg from the stream
blob = client.grab_jpeg() blob = client.grab_jpeg()

10
hardware-specific-tests/picamera2/test_exposure_time_drift.py
View file

@ -1,5 +1,4 @@
""" """Check exposure times do not drift.
Check exposure times do not drift.
This can get very tedious. Recommend running pytest with -s option This can get very tedious. Recommend running pytest with -s option
to monitor progress. to monitor progress.
@ -19,8 +18,7 @@ logging.basicConfig(level=logging.DEBUG)
def _test_exposure_time_drift(desired_time): def _test_exposure_time_drift(desired_time):
""" """Capture 10 full resolution images and check that the exposure time remains constant.
Capture 10 full resolution images and check that the exposure time remains constant
This confirms that automatic exposure time adjustment is fully turned off This confirms that automatic exposure time adjustment is fully turned off
""" """
@ -64,8 +62,6 @@ def _test_exposure_time_drift(desired_time):
def test_exposure_time_drift(): def test_exposure_time_drift():
""" """Performs the exposure time test for a range of exposure time values."""
Performs the exposure time test for a range of exposure time values.
"""
for desired_time in [100, 1000, 10000]: for desired_time in [100, 1000, 10000]:
_test_exposure_time_drift(desired_time) _test_exposure_time_drift(desired_time)

6
hardware-specific-tests/picamera2/test_sensor_mode.py
View file

@ -1,3 +1,5 @@
"""Test changing and setting camerat modes on the Raspberry Picamera."""
import logging import logging
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
@ -12,9 +14,7 @@ logging.basicConfig(level=logging.DEBUG)
def test_sensor_mode(): def test_sensor_mode():
""" """Test capturing raw arrays in two different sensor modes."""
Test capturing raw arrays in two different sensor modes
"""
cam = StreamingPiCamera2() cam = StreamingPiCamera2()
server = ThingServer() server = ThingServer()
server.add_thing(cam, "/camera/") server.add_thing(cam, "/camera/")

42
hardware-specific-tests/picamera2/test_tuning.py
View file

@ -1,6 +1,4 @@
""" """Tests that check that a tuning file can be reloaded."""
Tests that check that a tuning file can be reloaded
"""
import os import os
@ -16,17 +14,13 @@ MODEL = Picamera2.global_camera_info()[0]["Model"]
def load_default_tuning(): def load_default_tuning():
""" """Return the default tuning file for the connected camera."""
Return the default tuning file for the connected camera.
"""
fname = f"{MODEL}.json" fname = f"{MODEL}.json"
return Picamera2.load_tuning_file(fname) return Picamera2.load_tuning_file(fname)
def generate_bad_tuning(): def generate_bad_tuning():
""" """Return a tuning file with an invalid version number to force an error when loaded."""
Return a tuning file with an invalid version number to force an error when loaded.
"""
default_tuning = load_default_tuning() default_tuning = load_default_tuning()
bad_tuning = default_tuning.copy() bad_tuning = default_tuning.copy()
bad_tuning["version"] = 999 bad_tuning["version"] = 999
@ -34,8 +28,7 @@ def generate_bad_tuning():
def print_tuning(read_file: bool = False): def print_tuning(read_file: bool = False):
""" """Print the path of the default tuning file from the the environment variable.
Print the path of the default tuning file from the the environment variable.
:param read_file: Boolean, set true to also print the file contents. :param read_file: Boolean, set true to also print the file contents.
@ -53,7 +46,8 @@ def print_tuning(read_file: bool = False):
def _test_bad_tuning_after_good_tuning(configure: bool = False): def _test_bad_tuning_after_good_tuning(configure: bool = False):
""" """Test loading good, bad, then another good tuning files in sequence.
Load the default tuning file into the camera, re-load with a broken tuning file, Load the default tuning file into the camera, re-load with a broken tuning file,
check it errors. Finally check the default tuning file will load again afterwards. check it errors. Finally check the default tuning file will load again afterwards.
@ -101,19 +95,35 @@ def _test_bad_tuning_after_good_tuning(configure: bool = False):
@pytest.mark.filterwarnings("ignore: Exception ignored") @pytest.mark.filterwarnings("ignore: Exception ignored")
def test_bad_tuning_after_good_tuning_noconfigure(): def test_bad_tuning_after_good_tuning_noconfigure():
_test_bad_tuning_after_good_tuning(False) """First run setting good, then bad, then good tuning.
create_preview_configuration() is NOT run on initial setup.
"""
_test_bad_tuning_after_good_tuning(configure=False)
@pytest.mark.filterwarnings("ignore: Exception ignored") @pytest.mark.filterwarnings("ignore: Exception ignored")
def test_bad_tuning_after_good_tuning_configure(): def test_bad_tuning_after_good_tuning_configure():
_test_bad_tuning_after_good_tuning(True) """Second run setting good, then bad, then good tuning.
create_preview_configuration() is run on initial setup this time.
"""
_test_bad_tuning_after_good_tuning(configure=True)
@pytest.mark.filterwarnings("ignore: Exception ignored") @pytest.mark.filterwarnings("ignore: Exception ignored")
def test_bad_tuning_after_good_tuning_noconfigure2(): def test_bad_tuning_after_good_tuning_noconfigure2():
_test_bad_tuning_after_good_tuning(False) """3rd run setting good, then bad, then good tuning.
create_preview_configuration() is NOT run on initial setup.
"""
_test_bad_tuning_after_good_tuning(configure=False)
@pytest.mark.filterwarnings("ignore: Exception ignored") @pytest.mark.filterwarnings("ignore: Exception ignored")
def test_bad_tuning_after_good_tuning_configure2(): def test_bad_tuning_after_good_tuning_configure2():
_test_bad_tuning_after_good_tuning(True) """Final run setting good, then bad, then good tuning.
create_preview_configuration() is run on initial setup again.
"""
_test_bad_tuning_after_good_tuning(configure=True)

25
integration-tests/testfile.py
View file

@ -1,11 +1,10 @@
#! /usr/bin/env python3 #! /usr/bin/env python3
"""This module fires up a server in a subprocess to allow for integration tests. """Start a server subprocess for integration tests.
These are separated from the unit tests to stop them artificially inflating the test These tests are separated from unit tests to avoid inflating test coverage.
coverage. For now this file should be run directly not with a test framework. For now, this file should be run directly rather than through a test framework.
These tests are designed to run on CI. They should work on Linux or WSL for local They are designed to run on CI and should work on Linux or WSL for local debugging.
debugging.
""" """
from typing import Optional from typing import Optional
@ -31,7 +30,7 @@ SERVER_CMD: list[str] = [
def main() -> None: def main() -> None:
"""Set up the server, run basic checks, shutdown, check for graceful exit """Set up the server, run basic checks, shutdown, check for graceful exit.
The basic checks include checks that: The basic checks include checks that:
- The server boots - The server boots
@ -70,7 +69,7 @@ def main() -> None:
def test_client_connection() -> None: def test_client_connection() -> None:
"""Check a ThingClient can interact with the simulation microscope camera""" """Check a ThingClient can interact with the simulation microscope camera."""
print("Connecting Python client to microscope, and capturing image") print("Connecting Python client to microscope, and capturing image")
cam_client = lt.ThingClient.from_url("http://localhost:5000/camera/") cam_client = lt.ThingClient.from_url("http://localhost:5000/camera/")
img = Image.open(cam_client.grab_jpeg().open()) img = Image.open(cam_client.grab_jpeg().open())
@ -82,7 +81,7 @@ def test_client_connection() -> None:
def subscribe_to_mjpeg_stream() -> subprocess.Popen: def subscribe_to_mjpeg_stream() -> subprocess.Popen:
"""Start a background process subscribed to the mjpeg stream. """Start a background process subscribed to the mjpeg stream.
:return: The Popen object for the ongoing process. :returns: The Popen object for the ongoing process.
:raises: RuntimeError if the stream is not still connected after 2s :raises: RuntimeError if the stream is not still connected after 2s
""" """
@ -125,12 +124,11 @@ def set_up_working_dir() -> None:
def start_server() -> subprocess.Popen: def start_server() -> subprocess.Popen:
""" """Start the server in a subprocess.
Start the server in a subprocess.
The server is started in a subprocess and all outputs are buffered. The server is started in a subprocess and all outputs are buffered.
:return: Popen object for the ongoing process :returns: Popen object for the ongoing process
""" """
process = subprocess.Popen( process = subprocess.Popen(
SERVER_CMD, SERVER_CMD,
@ -156,7 +154,6 @@ def error_if_server_not_started(
:raises RuntimeError: If the server is not running as expected. :raises RuntimeError: If the server is not running as expected.
""" """
confirmed_uvicorn_is_running = False confirmed_uvicorn_is_running = False
t_start = time() t_start = time()
@ -181,7 +178,7 @@ def error_if_server_not_started(
def check_for_graceful_shutdown(server_process: subprocess.Popen) -> None: def check_for_graceful_shutdown(server_process: subprocess.Popen) -> None:
"""Check the server shutdown gracefully """Check the server shutdown gracefully.
Check the subprocess is not running Check the subprocess is not running
Check the logs have no errors Check the logs have no errors
@ -204,7 +201,7 @@ def check_for_graceful_shutdown(server_process: subprocess.Popen) -> None:
def read_process_buffers(process: subprocess.Popen) -> list[str]: def read_process_buffers(process: subprocess.Popen) -> list[str]:
"""Return STDOUT from a process""" """Return STDOUT from a process."""
stdout = [] stdout = []
while line := process.stdout.readline(): while line := process.stdout.readline():

19
pyproject.toml
View file

@ -40,6 +40,8 @@ dev = [
"pytest-mock==3.14", "pytest-mock==3.14",
"matplotlib~=3.10", "matplotlib~=3.10",
"hypothesis", "hypothesis",
"pydoctor~=25.4",
"codespell~=2.4",
] ]
pi = [ pi = [
"picamera2~=0.3.27", "picamera2~=0.3.27",
@ -106,10 +108,10 @@ select = [
# "B", # Flake8 bugbear # "B", # Flake8 bugbear
"A", # Flake8 builtins checker "A", # Flake8 builtins checker
"C", # Flake8 comprehensions "C", # Flake8 comprehensions
# "FIX", #TODO/FIXME's in code. These should be added during develoment for ongoing tasks. # "FIX", #TODO/FIXME's in code. These should be added during development for ongoing tasks.
# If they are not fixed before merge they should be converted to GitLab issues # If they are not fixed before merge they should be converted to GitLab issues
# "LOG", # Flake8 logging issues (pedantic logger formatting issues can be added with "G") # "LOG", # Flake8 logging issues (pedantic logger formatting issues can be added with "G")
# "T20", # Warns for print statments, production code should log # "T20", # Warns for print statements, production code should log
# "PT", # pytest linting # "PT", # pytest linting
"RET", # Consistent clear return statements "RET", # Consistent clear return statements
"RSE", # Raise parentheses "RSE", # Raise parentheses
@ -118,10 +120,19 @@ select = [
"C90", # McCabe complexity! "C90", # McCabe complexity!
"NPY", # Numpy linting "NPY", # Numpy linting
"N", # PEP8 naming "N", # PEP8 naming
# "D", # Docstring checks these may need to be added gradually "D", # Docstring checks these may need to be added gradually
"ERA001", # Commented out code! "ERA001", # Commented out code!
] ]
ignore = [ ignore = [
"E501" # Ignore long lines for now "E501", # Ignore long lines for now
"D203", # incompatible with D204
"D213", # incompatible with D212
"D400", # A stricter version of #415 that doesn't allow !
# The checkers below should be turned on as they complain about missing docstrings.
] ]
[tool.ruff.lint.pydocstyle]
# This lets the D401 checker understand that decorated thing properties and thing
# settings act like properties so should be documented as such.
property-decorators = ["labthings_fastapi.thing_property", "labthings_fastapi.thing_setting"]

4
ruff-increased-checking.toml
View file

@ -19,10 +19,10 @@ select = [
"B", # Flake8 bugbear "B", # Flake8 bugbear
"A", # Flake8 builtins checker "A", # Flake8 builtins checker
"C", # Flake8 comprehensions "C", # Flake8 comprehensions
"FIX", #TODO/FIXME's in code. These should be added during develoment for ongoing tasks. "FIX", #TODO/FIXME's in code. These should be added during development for ongoing tasks.
# If they are not fixed before merge they should be converted to GitLab issues # If they are not fixed before merge they should be converted to GitLab issues
"LOG", # Flake8 logging issues (pedantic logger formatting issues can be added with "G") "LOG", # Flake8 logging issues (pedantic logger formatting issues can be added with "G")
"T20", # Warns for print statments, production code should log "T20", # Warns for print statements, production code should log
"PT", # pytest linting "PT", # pytest linting
"RET", # Consistent clear return statements "RET", # Consistent clear return statements
"RSE", # Raise parentheses "RSE", # Raise parentheses

2
scripts/zenodo/README.md
View file

@ -1,5 +1,5 @@
# Zenodo archiving # Zenodo archiving
This folder contains some scripts put together by Kaspar Bumke, and re-used by Richard, to upload the contents of the repository, together with any CI build artefacts, to Zenodo. The result is a link, taking you to a pre-populated upload on Zenodo that you can manually correct and upload. In order to customise it for a new project, there are a few steps you need, outlined below. This folder contains some scripts put together by Kaspar Bumke, and reused by Richard, to upload the contents of the repository, together with any CI build artefacts, to Zenodo. The result is a link, taking you to a pre-populated upload on Zenodo that you can manually correct and upload. In order to customise it for a new project, there are a few steps you need, outlined below.
## Setting up archival ## Setting up archival
* Copy this folder to `scripts/zenodo` in your repository. * Copy this folder to `scripts/zenodo` in your repository.

12
scripts/zenodo/upload_to_zenodo.py
View file

@ -1,9 +1,11 @@
"""A script for uploading data to Zenodo."""
import os import os
from argparse import ArgumentParser, Namespace from argparse import ArgumentParser, Namespace
from zenodo import Zenodo from zenodo import Zenodo
import yaml import yaml
# you have to explicitely set ZENODO_USE_SANDBOX=false to not use the # You have to explicitly set ZENODO_USE_SANDBOX=false to not use the
# sandbox, any other value or unset variable means this script will use # sandbox, any other value or unset variable means this script will use
# the sandbox zenodo site # the sandbox zenodo site
if "ZENODO_USE_SANDBOX" in os.environ: if "ZENODO_USE_SANDBOX" in os.environ:
@ -19,17 +21,19 @@ else:
def parse_arguments() -> Namespace: def parse_arguments() -> Namespace:
"""Parse the commandline arguments and return a namespace containing the result."""
p = ArgumentParser(description="Upload data to Zenodo") p = ArgumentParser(description="Upload data to Zenodo")
p.add_argument("paths", help="Directories and files to upload to Zenodo", nargs="*") p.add_argument("paths", help="Directories and files to upload to Zenodo", nargs="*")
return p.parse_args() return p.parse_args()
def script_directory(path): def script_directory(path):
"""resolves path to directory of the current script""" """Return path to directory of the current script."""
return os.path.join(os.path.dirname(os.path.realpath(__file__)), path) return os.path.join(os.path.dirname(os.path.realpath(__file__)), path)
def get_meta(): def get_meta():
"""Return the metadata in the script directory as a dictionary."""
with open(script_directory("metadata.yaml"), encoding="utf-8") as f: with open(script_directory("metadata.yaml"), encoding="utf-8") as f:
metadata = f.read() metadata = f.read()
@ -37,6 +41,10 @@ def get_meta():
def main(): def main():
"""Create the zenodo deposit and upload it to Zenodo.
This is the main function called when the full script is run.
"""
args = parse_arguments() args = parse_arguments()
metadata = get_meta() metadata = get_meta()

30
scripts/zenodo/zenodo.py
View file

@ -1,9 +1,9 @@
""" """A script for packaging data for Zenodo.
Copyright (c) 2020 Bath Open Instrumentation Group Copyright (c) 2020 Bath Open Instrumentation Group
Adapted from: https://gitlab.com/schlauch/zenodo-api-test/ Adapted from: https://gitlab.com/schlauch/zenodo-api-test/
Copyright (c) 2018 German Aerospace Center (DLR). All rights reserved. Copyright (c) 2018 German Aerospace Center (DLR). All rights reserved.
SPDX-License-Identifier: MIT-DLR SPDX-License-Identifier: MIT-DLR
""" """
import json import json
@ -13,7 +13,15 @@ import requests
class Zenodo: class Zenodo:
"""A class that packages data for Zenodo."""
def __init__(self, api_token, use_sandbox=True): def __init__(self, api_token, use_sandbox=True):
"""Initialise the zenodo packager with the API token.
:param api_token: Your Zenodo API key.
:param use_sandbox: True to pushing to ``sandbox.zenodo.org`` rather than
``zenodo.org`` for testing.
"""
self._api_token = api_token self._api_token = api_token
self._use_sandbox = use_sandbox self._use_sandbox = use_sandbox
if use_sandbox: if use_sandbox:
@ -22,8 +30,7 @@ class Zenodo:
self.zenodo_url = "https://zenodo.org/api/deposit/depositions" self.zenodo_url = "https://zenodo.org/api/deposit/depositions"
def create_new_deposit(self): def create_new_deposit(self):
"""Creates a new (unpublished) Zenodo deposit and return its deposition ID.""" """Create a new (unpublished) Zenodo deposit and return its deposition ID."""
headers = {"Content-Type": "application/json"} headers = {"Content-Type": "application/json"}
r = requests.post( r = requests.post(
self.zenodo_url, self.zenodo_url,
@ -36,8 +43,7 @@ class Zenodo:
return r.json() return r.json()
def set_metadata(self, deposition_id, metadata): def set_metadata(self, deposition_id, metadata):
"""Sets the given metadata for the specified deposit.""" """Set the given metadata for the specified deposit."""
headers = {"Content-Type": "application/json"} headers = {"Content-Type": "application/json"}
r = requests.put( r = requests.put(
self.zenodo_url + "/{}".format(deposition_id), self.zenodo_url + "/{}".format(deposition_id),
@ -49,8 +55,7 @@ class Zenodo:
print(r.json()) print(r.json())
def upload_file(self, deposition_id, file_path): def upload_file(self, deposition_id, file_path):
"""Uploads a new file for the given deposit.""" """Upload a new file for the given deposit."""
file_name = os.path.basename(file_path) file_name = os.path.basename(file_path)
data = {"filename": file_name} data = {"filename": file_name}
files = {"file": open(file_path, "rb")} files = {"file": open(file_path, "rb")}
@ -64,8 +69,7 @@ class Zenodo:
print(r.json()) print(r.json())
def publish_deposit(self, deposition_id): def publish_deposit(self, deposition_id):
"""Publishes the given deposit. BEWARE: It is now visible to all!!!""" """Publish the given deposit. BEWARE: It is now visible to all!!!"""
r = requests.post( r = requests.post(
self.zenodo_url + "/{}/actions/publish".format(deposition_id), self.zenodo_url + "/{}/actions/publish".format(deposition_id),
params={"access_token": self._api_token}, params={"access_token": self._api_token},
@ -74,8 +78,7 @@ class Zenodo:
print(r.json()) print(r.json())
def create_new_version(self, deposition_id): def create_new_version(self, deposition_id):
"""Creates a new version of an already published deposit.""" """Create a new version of an already published deposit."""
r = requests.post( r = requests.post(
self.zenodo_url + "/{}/actions/newversion".format(deposition_id), self.zenodo_url + "/{}/actions/newversion".format(deposition_id),
params={"access_token": self._api_token}, params={"access_token": self._api_token},
@ -85,8 +88,7 @@ class Zenodo:
return os.path.basename(r.json()["links"]["latest_draft"]) return os.path.basename(r.json()["links"]["latest_draft"])
def remove_all_files(self, deposition_id): def remove_all_files(self, deposition_id):
"""Removes all uploaded files of a unpublished deposit.""" """Remove all uploaded files of a unpublished deposit."""
r = requests.get( r = requests.get(
self.zenodo_url + "/{}/files".format(deposition_id), self.zenodo_url + "/{}/files".format(deposition_id),
params={"access_token": self._api_token}, params={"access_token": self._api_token},

17
src/openflexure_microscope_server/__init__.py
View file

@ -0,0 +1,17 @@
"""The OpenFlexure Microscope Server Package.
The OpenFlexure Microscope Server is a LabThings-FastAPI server. All hardware
interfaces, and almost all functionality exposed over HTTP is done using ``Things``.
On the microscope the server is started by systemd. It can be controlled using commands
such as ``ofm restart``, ``ofm start``, and ``ofm stop``.
The server can also be run in simulation mode from the terminal. In the root directory
of the repository the server can be started with run:
.. code-block:: bash
openflexure-microscope-server --fallback -c ./ofm_config_simulation.json
"""

68
src/openflexure_microscope_server/logging.py
View file

@ -1,3 +1,16 @@
"""Configure logging for the OpenFlexure Server.
A module containing a custom logging handler and helper function for
the OpenFlexure server. These handle formatting logs, ensuring that logs
are logged both to a file and to the terminal (see note below!). Functionality
is also provided for retrieving logs for to send over HTTP.
Note that when running in production OpenFlexure Microscope the server is run via
``systemd``. As such, the logs that are sent to the terminal (STDERR) or any other
output to STDOUT/STDERR are captured by ``systemd``. These can be viewed by using
``journalctl`` or the ``ofm journal`` command.
"""
import logging import logging
from logging.handlers import RotatingFileHandler from logging.handlers import RotatingFileHandler
import os import os
@ -8,9 +21,17 @@ OFM_LOG_FILE = None
def configure_logging(log_folder): def configure_logging(log_folder):
"""Configure logging for the server while it is running.
This modifies the root logger to have a rotating file handler and
adds a custom handler that prints and stores all records except
``uvicorn.access`` logs.
It is important not to let Uvicorn override these settings.
"""
root_logger = logging.getLogger() root_logger = logging.getLogger()
root_logger.setLevel(logging.INFO) root_logger.setLevel(logging.INFO)
# Explictly make OFM_LOG_FILE a global so it can be updated based on log settings # Explicitly make OFM_LOG_FILE a global so it can be updated based on log settings
global OFM_LOG_FILE global OFM_LOG_FILE
OFM_LOG_FILE = os.path.join(log_folder, "openflexure_microscope.log") OFM_LOG_FILE = os.path.join(log_folder, "openflexure_microscope.log")
@ -39,25 +60,24 @@ def configure_logging(log_folder):
def retrieve_log() -> PlainTextResponse: def retrieve_log() -> PlainTextResponse:
""" """Return logs since the server started, up to a maximum of 250 messages.
Returns logs since we started running the server, up to a maximum of
250 messages. This log is the one shown in the UI and on the logging page.
It does not include any of the `uvicorn.access` logs as these are emmitted This log is the one shown in the UI and on the logging page.
It does not include any of the ``uvicorn.access`` logs as these are emitted
every time any API route is called. every time any API route is called.
All logs, including `uvicorn.access` are logged to the OFM_LOG_FILE (see above) All logs, including ``uvicorn.access`` are logged to the OFM_LOG_FILE (see above)
ths is the best place to get logs about crashes. this is the best place to get logs about crashes.
""" """
return PlainTextResponse(OFM_HANDLER.log_history) return PlainTextResponse(OFM_HANDLER.log_history)
def retrieve_log_from_file() -> PlainTextResponse: def retrieve_log_from_file() -> PlainTextResponse:
""" """Return the full log from the file for downloading.
Returns the full log from the file for downloading.
Note this is read and then sent as otherwise it causes a RuntimeError if it Note this is read and then sent as otherwise it causes a RuntimeError if it
is written to while sending through FileResponse is written to while sending through FileResponse.
""" """
if OFM_LOG_FILE is None: if OFM_LOG_FILE is None:
raise RuntimeError( raise RuntimeError(
@ -69,29 +89,33 @@ def retrieve_log_from_file() -> PlainTextResponse:
class OFMHandler(logging.Handler): class OFMHandler(logging.Handler):
""" """A logging.Handler that stores the most recent logs for access by the server."""
A child class of logging.Handler. This class handles storing the most recent
logs for access by the server.
"""
def __init__(self, level=logging.INFO, max_logs=250): def __init__(self, level=logging.INFO, max_logs=250):
"""Initialise the handler with a set logging level and message buffer size.
:param level: The level of logs captured. As standard logs of INFO and above
are captured.
:param max_logs: The maximum number of logs to hold in memory. This sets
how many can be returned over HTTP.
"""
super().__init__(level=level) super().__init__(level=level)
self._log = [] self._log = []
self._max_logs = max_logs self._max_logs = max_logs
def append_record(self, record): def append_record(self, record):
""" """Format message and append it to a list of records.
Use the built in formatter to format the record, then save
it to an array. Pop any in excess of the maximum number of logs The built in formatter is used to format the record.
Any records in excess of the maximum number of logs are removed.
""" """
self._log.append(self.format(record)) self._log.append(self.format(record))
while len(self._log) > self._max_logs: while len(self._log) > self._max_logs:
self._log.pop(0) self._log.pop(0)
def emit(self, record): def emit(self, record):
""" """Emit will save the logged record to the log."""
Emit will save the logged record to the log
"""
# Basic filter for now that simply stops uvicorn.access logs # Basic filter for now that simply stops uvicorn.access logs
# These are the logs each time an API endpoint is accessed # These are the logs each time an API endpoint is accessed
# This is only the log for the UI. # This is only the log for the UI.
@ -109,9 +133,7 @@ class OFMHandler(logging.Handler):
@property @property
def log_history(self): def log_history(self):
""" """Return the log history up to the maximum number of logs."""
Return the log history up to the maximum number of logs
"""
return "\n".join(self._log) return "\n".join(self._log)

102
src/openflexure_microscope_server/scan_directories.py
View file

@ -1,9 +1,4 @@
""" """Functionality to manage file system operations for scan directories."""
This submodule contains functionality for interacting with scan directories.
Currently it handles scan getting information from a information from a scan directory,
eventually is should handle all the file system operation for smart_scan.
"""
from typing import Optional from typing import Optional
import os import os
@ -21,11 +16,11 @@ IMAGE_REGEX = re.compile(r"-?[0-9]+_-?[0-9]+\.jpe?g$")
class NotEnoughFreeSpaceError(IOError): class NotEnoughFreeSpaceError(IOError):
pass """An exception raised if there is not enough free space on disk to scan."""
class ScanInfo(BaseModel): class ScanInfo(BaseModel):
"""Summary information about a scan folder""" """Summary information about a scan folder."""
name: str name: str
created: float created: float
@ -36,35 +31,37 @@ class ScanInfo(BaseModel):
class ScanDirectoryManager: class ScanDirectoryManager:
""" """A class for managing interactions with scan directories."""
A class for managing interactions with scan directories
"""
_base_scan_dir: str _base_scan_dir: str
def __init__(self, base_scan_dir: str): def __init__(self, base_scan_dir: str):
"""Initialise the scan directory manager.
:param base_scan_dir: Path of the directory that holds all scans.
"""
self._base_scan_dir = base_scan_dir self._base_scan_dir = base_scan_dir
if not os.path.exists(self._base_scan_dir): if not os.path.exists(self._base_scan_dir):
os.makedirs(self._base_scan_dir) os.makedirs(self._base_scan_dir)
@property @property
def base_dir(self) -> str: def base_dir(self) -> str:
"""The base directory scans are saved to""" """The base directory scans are saved to."""
return self._base_scan_dir return self._base_scan_dir
def exists(self, scan_name: str) -> bool: def exists(self, scan_name: str) -> bool:
"""True if scan of this name exists on disk""" """Return True if scan of this name exists on disk."""
return os.path.isdir(self.path_for(scan_name)) return os.path.isdir(self.path_for(scan_name))
def path_for(self, scan_name: str) -> str: def path_for(self, scan_name: str) -> str:
"""Return the path for a given scan name """Return the path for a given scan name.
Returns the path even if it doesn't exist Returns the path even if it doesn't exist
""" """
return os.path.join(self._base_scan_dir, scan_name) return os.path.join(self._base_scan_dir, scan_name)
def img_dir_for(self, scan_name: str) -> str: def img_dir_for(self, scan_name: str) -> str:
"""Return the path for the image dir for a given scan name """Return the path for the image dir for a given scan name.
Returns the path even if it doesn't exist Returns the path even if it doesn't exist
""" """
@ -73,7 +70,7 @@ class ScanDirectoryManager:
def get_file_path_from( def get_file_path_from(
self, scan_name: str, filename: str, check_exists: bool = False self, scan_name: str, filename: str, check_exists: bool = False
) -> Optional[str]: ) -> Optional[str]:
"""Return the full file path for the file within a scan directory """Return the full file path for the file within a scan directory.
If check_exists is True then None will be returned if the file does If check_exists is True then None will be returned if the file does
not exist. not exist.
@ -87,7 +84,7 @@ class ScanDirectoryManager:
def get_file_path_from_img_dir( def get_file_path_from_img_dir(
self, scan_name: str, filename: str, check_exists: bool = False self, scan_name: str, filename: str, check_exists: bool = False
) -> Optional[str]: ) -> Optional[str]:
"""Return the full file path for the file within a scan directory """Return the full file path for the file within a scan directory.
If check_exists is True, None is returned if the file does not exist. If False If check_exists is True, None is returned if the file does not exist. If False
then the path is returned anyway then the path is returned anyway
@ -110,19 +107,18 @@ class ScanDirectoryManager:
@property @property
def all_scans(self) -> list[str]: def all_scans(self) -> list[str]:
"""Return a list of the scan names in the base directory""" """Return a list of the scan names in the base directory."""
return [f.name for f in os.scandir(self._base_scan_dir) if f.is_dir()] return [f.name for f in os.scandir(self._base_scan_dir) if f.is_dir()]
def all_scans_info(self) -> list[ScanInfo]: def all_scans_info(self) -> list[ScanInfo]:
"""Return a lists of ScanInfo objects for each scan""" """Return a lists of ScanInfo objects for each scan."""
all_info: list[ScanInfo] = [] all_info: list[ScanInfo] = []
for scan_name in self.all_scans: for scan_name in self.all_scans:
all_info.append(ScanDirectory(scan_name, self.base_dir).scan_info()) all_info.append(ScanDirectory(scan_name, self.base_dir).scan_info())
return all_info return all_info
def _unique_scan_name(self, scan_name: str) -> str: def _unique_scan_name(self, scan_name: str) -> str:
"""Get the next unique scan name starting with the given name """Get the next unique scan name starting with the given name.
For more explanation on the scan naming see `new_scan_dir` For more explanation on the scan naming see `new_scan_dir`
""" """
@ -151,9 +147,9 @@ class ScanDirectoryManager:
return f"{scan_name}_{scan_num:0{SCAN_ZERO_PAD_DIGITS}d}" return f"{scan_name}_{scan_num:0{SCAN_ZERO_PAD_DIGITS}d}"
def new_scan_dir(self, scan_name: str) -> "ScanDirectory": def new_scan_dir(self, scan_name: str) -> "ScanDirectory":
"""Get a unique name for this scan and create a directory for it """Get a unique name for this scan and create a directory for it.
The scan will be named `{scan_name}_0001` where the number is The scan will be named ``{scan_name}_0001`` where the number is
zero-padded to be SCAN_ZERO_PAD_DIGITS digits long (to allow correct sorting if the zero-padded to be SCAN_ZERO_PAD_DIGITS digits long (to allow correct sorting if the
scans are ordered alphanumerically). scans are ordered alphanumerically).
@ -161,7 +157,6 @@ class ScanDirectoryManager:
Returns the a ScanDirectory object Returns the a ScanDirectory object
""" """
# if no scan name is set set to "scan". This done here as empty strings # if no scan name is set set to "scan". This done here as empty strings
# get passed in otherwise. # get passed in otherwise.
if not scan_name: if not scan_name:
@ -174,26 +169,25 @@ class ScanDirectoryManager:
return ScanDirectory(full_scan_name, self.base_dir) return ScanDirectory(full_scan_name, self.base_dir)
def delete_scan(self, scan_name: str) -> None: def delete_scan(self, scan_name: str) -> None:
"""Delete a scan""" """Delete a scan."""
shutil.rmtree(self.path_for(scan_name)) shutil.rmtree(self.path_for(scan_name))
def zip_scan(self, scan_name: str, final_version: bool = False) -> "ScanDirectory": def zip_scan(self, scan_name: str, final_version: bool = False) -> "ScanDirectory":
"""Zips any images from the scan not yet zipped, return full path to zip """Zips any images from the scan not yet zipped, return full path to zip.
`final_version` Set true to stitch all files not just the scan images ``final_version`` Set true to stitch all files not just the scan images
this should only be done at the end as it is not possible to update a file this should only be done at the end as it is not possible to update a file
in a zip. in a zip.
""" """
return ScanDirectory(scan_name, self.base_dir).zip_files(final_version) return ScanDirectory(scan_name, self.base_dir).zip_files(final_version)
def check_free_disk_space(self, min_space: int = 500000000) -> None: def check_free_disk_space(self, min_space: int = 500000000) -> None:
""" """Raise an exception if there is not enough free disk space to continue scanning.
Raise an exception if there is not enough free disk space to continue scanning
:param min_space: the minimum space required in bytes. :param min_space: the minimum space required in bytes.
Default = 500,000,000 (500MB) Default = 500,000,000 (500MB)
:raises: NotEnoughFreeSpaceError if the remaining storage is below min_space :raises NotEnoughFreeSpaceError: if the remaining storage is below min_space
""" """
disk_usage = shutil.disk_usage(self._base_scan_dir) disk_usage = shutil.disk_usage(self._base_scan_dir)
if disk_usage.free < min_space: if disk_usage.free < min_space:
@ -204,17 +198,17 @@ class ScanDirectoryManager:
class ScanDirectory: class ScanDirectory:
"""A class for handling interactions with scan directories """A class for handling interactions with scan directories."""
Initalisation parameters:
name: the name of the scan (the scan directory basename)
base_scan_dir: Path of the directory that holds all scans
"""
_name: str _name: str
_base_scan_dir: str _base_scan_dir: str
def __init__(self, name: str, base_scan_dir: str): def __init__(self, name: str, base_scan_dir: str):
"""Initialise the scan directory.
:param name: the name of the scan (the scan directory basename).
:param base_scan_dir: Path of the directory that holds all scans.
"""
self._name = name self._name = name
self._base_scan_dir = base_scan_dir self._base_scan_dir = base_scan_dir
if not os.path.isdir(self.dir_path): if not os.path.isdir(self.dir_path):
@ -224,19 +218,19 @@ class ScanDirectory:
@property @property
def name(self) -> str: def name(self) -> str:
"""The name of the scan""" """The name of the scan."""
return self._name return self._name
@property @property
def dir_path(self) -> str: def dir_path(self) -> str:
"""The full path to the scan directory""" """The full path to the scan directory."""
return os.path.join(self._base_scan_dir, self._name) return os.path.join(self._base_scan_dir, self._name)
@property @property
def images_dir(self) -> Optional[str]: def images_dir(self) -> Optional[str]:
""" """The path to the images directory.
The path to the images directory. None is returned if no images directory
was created None is returned if no images directory was created.
""" """
im_path = os.path.join(self.dir_path, IMG_DIR_NAME) im_path = os.path.join(self.dir_path, IMG_DIR_NAME)
if os.path.isdir(im_path): if os.path.isdir(im_path):
@ -245,27 +239,27 @@ class ScanDirectory:
@property @property
def created_time(self) -> float: def created_time(self) -> float:
"""The time the directory was created on disk""" """The time the directory was created on disk."""
return os.path.getctime(self.dir_path) return os.path.getctime(self.dir_path)
def get_scan_files(self): def get_scan_files(self):
"""Return a list of the files in the images dir""" """Return a list of the files in the images dir."""
if self.images_dir is None: if self.images_dir is None:
return [] return []
return os.listdir(self.images_dir) return os.listdir(self.images_dir)
def _extract_scan_images(self, file_list: list[str]): def _extract_scan_images(self, file_list: list[str]):
"""Extract files which match the naming convention for scan images """Extract files which match the naming convention for scan images.
:param file_list: The list of files to search. Normally this would be :param file_list: The list of files to search. Normally this would be
`self.get_scan_files()` ``self.get_scan_files()``
:returns: The list of files that match the naming convention for scan images :returns: The list of files that match the naming convention for scan images
""" """
return [i for i in file_list if IMAGE_REGEX.search(i)] return [i for i in file_list if IMAGE_REGEX.search(i)]
def _extract_final_stitches(self, file_list: list[str]): def _extract_final_stitches(self, file_list: list[str]):
"""Extract files which match the naming convention for final stitches """Extract files which match the naming convention for final stitches.
:param file_list: The list of files to search. :param file_list: The list of files to search.
@ -274,7 +268,7 @@ class ScanDirectory:
return [i for i in file_list if STITCH_REGEX.search(i)] return [i for i in file_list if STITCH_REGEX.search(i)]
def _extract_dzi_files(self, file_list: list[str]): def _extract_dzi_files(self, file_list: list[str]):
"""Extract files which match the naming convention for dzi_files """Extract files which match the naming convention for dzi_files.
:param file_list: The list of files to search. :param file_list: The list of files to search.
@ -283,7 +277,7 @@ class ScanDirectory:
return [i for i in file_list if i.endswith("dzi")] return [i for i in file_list if i.endswith("dzi")]
def get_final_stitch_name(self) -> Optional[str]: def get_final_stitch_name(self) -> Optional[str]:
"""Return the filename for the final stitch (in the images dir) """Return the filename for the final stitch (in the images dir).
If no final stitch is found, return None If no final stitch is found, return None
""" """
@ -293,12 +287,11 @@ class ScanDirectory:
return stitches[0] return stitches[0]
def get_modified_time(self) -> float: def get_modified_time(self) -> float:
"""Return the modified time of the directory""" """Return the modified time of the directory."""
return max(os.stat(root).st_mtime for root, _, _ in os.walk(self.dir_path)) return max(os.stat(root).st_mtime for root, _, _ in os.walk(self.dir_path))
def scan_info(self) -> ScanInfo: def scan_info(self) -> ScanInfo:
"""Return the information for the scan directory as a ScanInfo object""" """Return the information for the scan directory as a ScanInfo object."""
scan_files = self.get_scan_files() scan_files = self.get_scan_files()
scan_images = self._extract_scan_images(scan_files) scan_images = self._extract_scan_images(scan_files)
stitches = self._extract_final_stitches(scan_files) stitches = self._extract_final_stitches(scan_files)
@ -336,13 +329,12 @@ class ScanDirectory:
return files return files
def zip_files(self, final_version: bool = False) -> str: def zip_files(self, final_version: bool = False) -> str:
"""Zips any images from the scan not yet zipped, return full path to zip """Zips any images from the scan not yet zipped, return full path to zip.
`final_version` Set true to stitch all files not just the scan images ``final_version`` Set true to stitch all files not just the scan images
this should only be done at the end as it is not possible to update a file this should only be done at the end as it is not possible to update a file
in a zip. in a zip.
""" """
zip_fname = os.path.join(self.dir_path, "images.zip") zip_fname = os.path.join(self.dir_path, "images.zip")
if os.path.isfile(zip_fname): if os.path.isfile(zip_fname):
@ -369,6 +361,6 @@ class ScanDirectory:
def get_files_in_zip(zip_path): def get_files_in_zip(zip_path):
"""List the relative paths of all files and folders in the zip folder specified""" """List the relative paths of all files and folders in the zip folder specified."""
scan_zip = zipfile.ZipFile(zip_path) scan_zip = zipfile.ZipFile(zip_path)
return [os.path.normpath(i) for i in scan_zip.namelist()] return [os.path.normpath(i) for i in scan_zip.namelist()]

188
src/openflexure_microscope_server/scan_planners.py
View file

@ -1,5 +1,4 @@
""" """Functionality for planning scan routes.
This module contains functionality for planning a scan route
A scan route can be planned by a ScanPlanner class currently there A scan route can be planned by a ScanPlanner class currently there
is only one type the SmartSpiral. More can be added using by is only one type the SmartSpiral. More can be added using by
@ -27,8 +26,11 @@ NEIGHBOUR_CUTOFF = 1.4
def enforce_xy_tuple(value: XYPos) -> XYPos: def enforce_xy_tuple(value: XYPos) -> XYPos:
""" """Check input is a tuple and is of length 2.
Used for enforcing that an input is a tuple and is the correct length
If possible it will coerce the value to a tuple.
:raises ValueError: if the input cannot be coerced to a tuple of length 2.
""" """
if not isinstance(value, (list, tuple)): if not isinstance(value, (list, tuple)):
raise ValueError("2 value tuple expected") raise ValueError("2 value tuple expected")
@ -40,8 +42,11 @@ def enforce_xy_tuple(value: XYPos) -> XYPos:
def enforce_xyz_tuple(value: XYZPos) -> XYZPos: def enforce_xyz_tuple(value: XYZPos) -> XYZPos:
""" """Check input is a tuple and is of length 3.
Used for enforcing that an input is a tuple and is the correct length
If possible it will coerce the value to a tuple.
:raises ValueError: if the input cannot be coerced to a tuple of length 3.
""" """
if not isinstance(value, (list, tuple)): if not isinstance(value, (list, tuple)):
raise ValueError("3 value tuple expected") raise ValueError("3 value tuple expected")
@ -53,40 +58,39 @@ def enforce_xyz_tuple(value: XYZPos) -> XYZPos:
class ScanPlanner: class ScanPlanner:
""" """A base class for a scan planner.
A base class for a scan planner.
This should never be used directly for a scan, it should be subclassed. This should never be used directly for a scan, it should be subclassed.
Each subclass should implement at least the methods with NotImplementedError Each subclass should implement at least the methods with NotImplementedError
set: set:
* _parse() - to parse the planner_settings dictionary, saving values to class
variables
* _intial_location_list() - Sets the list of locations for the scan to follow
For a simple scan pattern this should be sufficent. For more complex ones that * ``_parse()`` - to parse the planner_settings dictionary, saving values to class
dynanically adjust the path it is suggested to override `mark_location_visited()` variables
calling `super().mark_location_visited()` at the start of the method so that all * ``_initial_location_list()`` - Sets the list of locations for the scan to follow
For a simple scan pattern this should be sufficient. For more complex ones that
dynamically adjust the path it is suggested to override ``mark_location_visited()``
calling ``super().mark_location_visited()`` at the start of the method so that all
locations are adjusted. locations are adjusted.
When subclassing be sure to use enforce_xy_tuple and enforce_xyz_tuple on any user When subclassing be sure to use ``enforce_xy_tuple`` and ``enforce_xyz_tuple`` on
data before running any user data before running.
""" """
def __init__(self, intial_position: XYPos, planner_settings: Optional[dict] = None): def __init__(
""" self, initial_position: XYPos, planner_settings: Optional[dict] = None
Set up lists for the path planning, and scan history. ):
""" """Set up lists for the path planning, and scan history."""
self._initial_position = enforce_xy_tuple(initial_position)
self._initial_position = enforce_xy_tuple(intial_position)
self._parse(planner_settings) self._parse(planner_settings)
# The remaining (x,y) locations to scan # The remaining (x,y) locations to scan
# (This was `path` before refactoring from the long `sample_scan` code) # (This was `path` before refactoring from the long `sample_scan` code)
self._remaining_locations: XYPosList = self._intial_location_list() self._remaining_locations: XYPosList = self._initial_location_list()
# This holds a list of all (x,y,z) locations where images were taken # This holds a list of all (x,y,z) locations where images were taken
# this may not be equivalent to the x,y poistions ins self._path_history # this may not be equivalent to the x,y positions ins self._path_history
# if background detect is used # if background detect is used
# (This was not used in the `sample_scan` code) # (This was not used in the `sample_scan` code)
self._imaged_locations: XYZPosList = [] self._imaged_locations: XYZPosList = []
@ -103,48 +107,37 @@ class ScanPlanner:
@property @property
def scan_complete(self) -> bool: def scan_complete(self) -> bool:
""" """Return True if there are no locations left to scan."""
Return True if there are no locations left to scan.
"""
return not self._remaining_locations return not self._remaining_locations
@property @property
def remaining_locations(self) -> XYPosList: def remaining_locations(self) -> XYPosList:
""" """Property to access a copy of the remaining_locations."""
Property to access a copy of the remaining_locations
"""
return copy(self._remaining_locations) return copy(self._remaining_locations)
@property @property
def imaged_locations(self) -> XYZPosList: def imaged_locations(self) -> XYZPosList:
""" """Property to access a copy of the imaged_locations."""
Property to access a copy of the imaged_locations
"""
return copy(self._imaged_locations) return copy(self._imaged_locations)
@property @property
def focused_locations(self) -> XYZPosList: def focused_locations(self) -> XYZPosList:
""" """Property to access a copy of the focused_locations."""
Property to access a copy of the focused_locations
"""
return copy(self._focused_locations) return copy(self._focused_locations)
@property @property
def path_history(self) -> XYPosList: def path_history(self) -> XYPosList:
""" """Property to access a copy of the path_history."""
Property to access a copy of the path_history
"""
return copy(self._path_history) return copy(self._path_history)
def _parse(self, planner_settings: Optional[dict] = None) -> None: def _parse(self, planner_settings: Optional[dict] = None) -> None:
""" """Parse any settings sent to this planner and store them if needed."""
Parse any settings sent to this planner and store them if needed.
"""
raise NotImplementedError("Did you call the ScanPlanner base class?") raise NotImplementedError("Did you call the ScanPlanner base class?")
def _intial_location_list(self) -> XYPosList: def _initial_location_list(self) -> XYPosList:
""" """Set the initial list of locations for this scan planner.
Called on initalisation. Sets the initial list of locations for this scan planner
This is called on initialisation.
For a simple grid scan/snake scan this would be all locations to move to. For a simple grid scan/snake scan this would be all locations to move to.
@ -154,23 +147,17 @@ class ScanPlanner:
raise NotImplementedError("Did you call the ScanPlanner base class?") raise NotImplementedError("Did you call the ScanPlanner base class?")
def position_visited(self, position: XYPos) -> bool: def position_visited(self, position: XYPos) -> bool:
""" """Return True if input xy position has been visited before."""
Return True if input xy position has been visited before
"""
# Ensure tuple for correct matching! # Ensure tuple for correct matching!
return tuple(position) in self._path_history return tuple(position) in self._path_history
def position_planned(self, position: XYPos) -> bool: def position_planned(self, position: XYPos) -> bool:
""" """Return True if input xy position is planned."""
Return True if input xy position is planned
"""
# Ensure tuple for correct matching! # Ensure tuple for correct matching!
return tuple(position) in self._remaining_locations return tuple(position) in self._remaining_locations
def get_next_location_and_z_estimate(self) -> tuple[XYPos, Optional[int]]: def get_next_location_and_z_estimate(self) -> tuple[XYPos, Optional[int]]:
""" """Return the next location to scan and its estimated z-position.
Return the next location to scan, and the estimated z-position
for this location.
Note z-position may be None! This indicates that the current z, position Note z-position may be None! This indicates that the current z, position
should be used. should be used.
@ -190,10 +177,12 @@ class ScanPlanner:
return next_location, z return next_location, z
def closest_focus_site(self, xy_pos: XYPos) -> Optional[XYZPos]: def closest_focus_site(self, xy_pos: XYPos) -> Optional[XYZPos]:
""" """Return the xyz position of the closest site where focus was achieved.
Return the xyz position of the closest site where focus was achieved
to the input xy_position, with the most recently taken image returned in The most recently taken image is returned in the case of a tie.
the case of a tie
:param xy_pos: The xy_position which the returned position should be closest
to.
Returns None if there if no focussed locations are present Returns None if there if no focussed locations are present
""" """
@ -218,13 +207,13 @@ class ScanPlanner:
def mark_location_visited( def mark_location_visited(
self, xyz_pos: XYZPos, imaged: bool, focused: bool self, xyz_pos: XYZPos, imaged: bool, focused: bool
) -> None: ) -> None:
""" """Mark the location as visited.
Mark the location as visited
Args: Args:
xyz_pos: the x_y_z position xyz_pos: the x_y_z position
imaged: true if an image was taken, false if not (due to background detect) imaged: true if an image was taken, false if not (due to background detect)
focused: true if autofocus completed successfully focused: true if autofocus completed successfully
""" """
# ensure is tuple! # ensure is tuple!
xyz_pos = enforce_xyz_tuple(xyz_pos) xyz_pos = enforce_xyz_tuple(xyz_pos)
@ -246,14 +235,14 @@ class ScanPlanner:
class SmartSpiral(ScanPlanner): class SmartSpiral(ScanPlanner):
""" """A scan planner that spirals outward from the centre, prioritising short moves.
This is a smart spiral scan that spirals out from the centre, but prioritises
short moves over rigidly sticking to minimising radius from the centre of the This planner spirals out from the centre, but prioritises short moves over rigidly
scan. sticking to minimising radius from the centre of the scan.
Each time and image is taken the four neighbouring images are added Each time and image is taken the four neighbouring images are added
to the list of poisitions to image (unless they are already listed or to the list of positions to image (unless they are already listed or
tried). However if a location is not imaged due no sample being detected tried). However, if a location is not imaged due no sample being detected
then neibouring positions are not imaged. then neibouring positions are not imaged.
The next image taken is the closes to the centre (considering the largest The next image taken is the closes to the centre (considering the largest
@ -266,14 +255,12 @@ class SmartSpiral(ScanPlanner):
_dy: int = 0 _dy: int = 0
def _parse(self, planner_settings: Optional[dict] = None) -> None: def _parse(self, planner_settings: Optional[dict] = None) -> None:
""" """Parse SmartSpiral Settings dictionary.
Parse SmartSpiral Settings. This should be a dictionary
"dx" - the movement size in x * ``dx`` - the movement size in x
"dy" - the movement size in y * ``dy`` - the movement size in y
"max_dist" - The maximum distance to a location can be from the centre. * ``max_dist`` - The maximum distance to a location can be from the centre.
""" """
expected_keys = ["max_dist", "dx", "dy"] expected_keys = ["max_dist", "dx", "dy"]
invalid_msg = "SmartSpiral requires a planner_settings dictionary with keys: " invalid_msg = "SmartSpiral requires a planner_settings dictionary with keys: "
if not planner_settings: if not planner_settings:
@ -285,9 +272,10 @@ class SmartSpiral(ScanPlanner):
self._dy = int(planner_settings["dy"]) self._dy = int(planner_settings["dy"])
self._max_dist = int(planner_settings["max_dist"]) self._max_dist = int(planner_settings["max_dist"])
def _intial_location_list(self) -> XYPosList: def _initial_location_list(self) -> XYPosList:
""" """Set the initial list of locations for this scan planner.
Called on initalisation. Sets the initial list of locations for this scan planner
This is salled on initialisation.
For smart spiral this is just the first point For smart spiral this is just the first point
""" """
@ -296,13 +284,13 @@ class SmartSpiral(ScanPlanner):
def mark_location_visited( def mark_location_visited(
self, xyz_pos: XYZPos, imaged: bool = True, focused: bool = True self, xyz_pos: XYZPos, imaged: bool = True, focused: bool = True
) -> None: ) -> None:
""" """Mark the location as visited. Adjust extra positions accordingly.
Mark the location as visited. Adjust extra positions accordingly
Args: Args:
xyz_pos: the x_y_z position xyz_pos: the x_y_z position
imaged: true if an image was taken, false if not (due to background detect) imaged: true if an image was taken, false if not (due to background detect)
focused: true if autofocus completed successfully focused: true if autofocus completed successfully
""" """
# First call the base class to update the positions # First call the base class to update the positions
super().mark_location_visited(xyz_pos, imaged, focused) super().mark_location_visited(xyz_pos, imaged, focused)
@ -313,10 +301,14 @@ class SmartSpiral(ScanPlanner):
self._re_sort_remaining_locations(xy_pos) self._re_sort_remaining_locations(xy_pos)
def _add_surrounding_positions(self, xy_pos: XYPos) -> None: def _add_surrounding_positions(self, xy_pos: XYPos) -> None:
""" """Add the 4 surrounding positions to the list of remaining locations to visit.
This adds the surrounding (4 point connectivity) positions
to the remaining locations if they are not too far away or This adds the surrounding positions (with 4 point connectivity) to the
already planned or already visited remaining locations list if they are not:
* too far away
* already planned
* already visited
""" """
new_positions = [ new_positions = [
(xy_pos[0] - self._dx, xy_pos[1]), (xy_pos[0] - self._dx, xy_pos[1]),
@ -337,9 +329,7 @@ class SmartSpiral(ScanPlanner):
self._remaining_locations.append(new_pos) self._remaining_locations.append(new_pos)
def _re_sort_remaining_locations(self, current_pos: XYPos) -> None: def _re_sort_remaining_locations(self, current_pos: XYPos) -> None:
""" """Sort the remaining positions based on the current location."""
Sort the remaining positions besed on the current location
"""
# Defined rather than use a lambda for readability # Defined rather than use a lambda for readability
def sort_key(pos): def sort_key(pos):
@ -352,10 +342,10 @@ class SmartSpiral(ScanPlanner):
self._remaining_locations.sort(key=sort_key) self._remaining_locations.sort(key=sort_key)
def get_next_location_and_z_estimate(self) -> tuple[XYPos, Optional[int]]: def get_next_location_and_z_estimate(self) -> tuple[XYPos, Optional[int]]:
""" """Return the next location to scan and its estimated z-position.
Return the next location to scan, and the estimated z-position
for this location. This overrides the default behaviour of ScanPlanner This overrides the default behaviour of ScanPlanner to take the lowest value of
to take the lowest value of nearest neighbours as this works best for smart stack nearest neighbours as this works best for smart stack.
Note z-position may be None! This indicates that the current z position Note z-position may be None! This indicates that the current z position
should be used. should be used.
@ -375,11 +365,12 @@ class SmartSpiral(ScanPlanner):
return next_location, z return next_location, z
def select_nearby_focus_site(self, xy_pos: XYPos) -> Optional[XYZPos]: def select_nearby_focus_site(self, xy_pos: XYPos) -> Optional[XYZPos]:
""" """Return the xyz position of the nearby site with the lowest z position.
Return the xyz position of the nearby site with the lowest z position.
Lowest position is best, as starting too high causes smart stacking to Lowest position is best, as starting too high causes smart stacking to
autofocus and restart. Starting too low just requires extra movements in +z. autofocus and restart. Starting too low just requires extra movements in +z.
Nearby is defined as within NEIGHBOUR_CUTOFF times the distance to the closest neighbour. Nearby is defined as within NEIGHBOUR_CUTOFF times the distance to the closest
neighbour.
Returns None if there if no focused locations are present Returns None if there if no focused locations are present
""" """
@ -413,13 +404,11 @@ class SmartSpiral(ScanPlanner):
starting_pos: XYPos | np.ndarray, starting_pos: XYPos | np.ndarray,
ending_pos: XYPos | np.ndarray, ending_pos: XYPos | np.ndarray,
) -> float: ) -> float:
""" """Return the larger of x moves or y moves between two xy positions.
Return the number of moves between two xy positions in the x or y direction
whichever is largest :param starting_pos: the position to measure from
:param ending_pos: the position to measure to
Args:
starting_pos: the position to measure from
ending_pos: the position to measure to
""" """
move_size = np.array([self._dx, self._dy]) move_size = np.array([self._dx, self._dy])
@ -434,10 +423,9 @@ class SmartSpiral(ScanPlanner):
def distance_between( def distance_between(
current_pos: XYPos | np.ndarray, next_pos: XYPos | np.ndarray current_pos: XYPos | np.ndarray, next_pos: XYPos | np.ndarray
) -> float: ) -> float:
""" """Calculate the distance between the two xy positions.
Calculate the distance between the two xy positions
This was previously called `distance_to_site` This was previously called ``distance_to_site``
""" """
next_pos = np.array(next_pos, dtype="float64") next_pos = np.array(next_pos, dtype="float64")
current_pos = np.array(current_pos, dtype="float64") current_pos = np.array(current_pos, dtype="float64")

16
src/openflexure_microscope_server/server/__init__.py
View file

@ -1,3 +1,5 @@
"""A package responsible for, setup, booting, and shutting down the server."""
from __future__ import annotations from __future__ import annotations
from typing import Optional, Callable from typing import Optional, Callable
@ -14,11 +16,10 @@ from ..logging import configure_logging, retrieve_log, retrieve_log_from_file
def set_shutdown_function(shutdown_function: Callable[[], None]): def set_shutdown_function(shutdown_function: Callable[[], None]):
""" """Ensure a function is called before the shutdown.
Ensure a function is called before the shutdown
This monkey patches the Uvicorn Server's handle_exit. This is needed because This monkey patches the Uvicorn Server's handle_exit. This is needed because
the uvicorn `lifecycle` events and FastAPI `shutdown` events only fire once the uvicorn ``lifecycle`` events and FastAPI ``shutdown`` events only fire once
background tasks have completed. background tasks have completed.
Without this the system exits cleanly only if no client is receiving a Without this the system exits cleanly only if no client is receiving a
@ -26,9 +27,8 @@ def set_shutdown_function(shutdown_function: Callable[[], None]):
send streaming responses. send streaming responses.
:param shutdown_function: A callable with no arguments or outputs. This :param shutdown_function: A callable with no arguments or outputs. This
should stop any async generators that may be sending to streaming responses. should stop any async generators that may be sending to streaming responses.
""" """
original_handler = Server.handle_exit original_handler = Server.handle_exit
@wraps(Server.handle_exit) @wraps(Server.handle_exit)
@ -53,12 +53,10 @@ def customise_server(
def _get_scans_dir(config: dict) -> Optional[str]: def _get_scans_dir(config: dict) -> Optional[str]:
""" """Read the config and return the scans directory.
Read the config and return the scans directory.
Return is None if there is no /smart_scan/ thing loaded. Return is None if there is no /smart_scan/ thing loaded.
""" """
if "/smart_scan/" in config["things"]: if "/smart_scan/" in config["things"]:
try: try:
return config["things"]["/smart_scan/"]["kwargs"]["scans_folder"] return config["things"]["/smart_scan/"]["kwargs"]["scans_folder"]
@ -69,7 +67,7 @@ def _get_scans_dir(config: dict) -> Optional[str]:
def serve_from_cli(argv: Optional[list[str]] = None): def serve_from_cli(argv: Optional[list[str]] = None):
"""Start the server from the command line""" """Start the server from the command line."""
args = lt.cli.parse_args(argv) args = lt.cli.parse_args(argv)
log_config = copy(uvicorn.config.LOGGING_CONFIG) log_config = copy(uvicorn.config.LOGGING_CONFIG)

12
src/openflexure_microscope_server/server/legacy_api.py
View file

@ -1,9 +1,12 @@
"""Provide endpoints that mimic the v2 API for OpenFlexure Connect discoverability."""
import labthings_fastapi as lt import labthings_fastapi as lt
from fastapi import Response from fastapi import Response
from socket import gethostname from socket import gethostname
def add_v2_endpoints(thing_server: lt.ThingServer): def add_v2_endpoints(thing_server: lt.ThingServer):
"""Add the v2 API endpoints for OpenFlexure Connect discoverability."""
app = thing_server.app app = thing_server.app
# TODO: update openflexure connect to make this unnecessary!! # TODO: update openflexure connect to make this unnecessary!!
@ -12,7 +15,10 @@ def add_v2_endpoints(thing_server: lt.ThingServer):
# This is necessary until Connect is rebuilt. # This is necessary until Connect is rebuilt.
@app.get("/routes") @app.get("/routes")
def routes_stub() -> dict[str, dict]: def routes_stub() -> dict[str, dict]:
"""A stub list of routes, used by OF Connect to identify the microscope""" """Return a stub list of routes.
This is used by OF Connect to identify the microscope.
"""
fake_routes = [ fake_routes = [
"/api/v2/", "/api/v2/",
"/api/v2/streams/snapshot", "/api/v2/streams/snapshot",
@ -26,11 +32,11 @@ def add_v2_endpoints(thing_server: lt.ThingServer):
@app.get("/api/v2/streams/snapshot") @app.get("/api/v2/streams/snapshot")
@app.head("/api/v2/streams/snapshot") @app.head("/api/v2/streams/snapshot")
async def thumbnail() -> JPEGResponse: async def thumbnail() -> JPEGResponse:
"""A low-resolution snapshot, for compatibility with OF connect""" """Return a low-resolution snapshot, for compatibility with OF connect."""
blob = await thing_server.things["/camera/"].lores_mjpeg_stream.grab_frame() blob = await thing_server.things["/camera/"].lores_mjpeg_stream.grab_frame()
return JPEGResponse(blob) return JPEGResponse(blob)
@app.get("/api/v2/instrument/settings/name") @app.get("/api/v2/instrument/settings/name")
def get_hostname() -> str: def get_hostname() -> str:
"""Get the hostname of the device, for compatibility with OF connect""" """Get the hostname of the device, for compatibility with OF connect."""
return gethostname() return gethostname()

11
src/openflexure_microscope_server/server/serve_static_files.py
View file

@ -1,3 +1,5 @@
"""Add endpoints for static files to the underlying FastAPI server."""
import os import os
from typing import Optional from typing import Optional
@ -9,9 +11,10 @@ THIS_DIR = os.path.dirname(os.path.abspath(__file__))
def add_static_file(app: FastAPI, fname: str, folder: str) -> None: def add_static_file(app: FastAPI, fname: str, folder: str) -> None:
"""Add a single file to the root of the FastAPI app """Add a single file to the root of the FastAPI app.
The file with name `fname` will be mounted at `/fname` - the
`folder` does not affect where it is mounted in the app. The file with name ``fname`` will be mounted at ``/fname`` - the
``folder`` does not affect where it is mounted in the app.
app: The FastAPI app to add to, in this case the OpenFlexure server app: The FastAPI app to add to, in this case the OpenFlexure server
fname: the name of the file to add fname: the name of the file to add
@ -24,7 +27,7 @@ def add_static_file(app: FastAPI, fname: str, folder: str) -> None:
def add_static_files(app: FastAPI, scans_folder: Optional[str]) -> None: def add_static_files(app: FastAPI, scans_folder: Optional[str]) -> None:
"""Add the static files responsible for the webapp app to the FastAPI app """Add the static files responsible for the webapp app to the FastAPI app.
app: The FastAPI app to add to, in this case the OpenFlexure server app: The FastAPI app to add to, in this case the OpenFlexure server
""" """

5
src/openflexure_microscope_server/things/__init__.py
View file

@ -0,0 +1,5 @@
"""A package for all of the core LabThings-FastAPI Things shipped with the microscope.
The microscope can be extended to be used with other hardware by creating a package
with other Things and including them in the LabThings-FastAPI config file.
"""

11
src/openflexure_microscope_server/things/auto_recentre_stage.py
View file

@ -1,3 +1,5 @@
"""Provide functionality for automatically recentring the Stage."""
import numpy as np import numpy as np
import logging import logging
@ -14,6 +16,12 @@ AutofocusDep = lt.deps.direct_thing_client_dependency(AutofocusThing, "/autofocu
class RecentringThing(lt.Thing): class RecentringThing(lt.Thing):
"""A Thing for recentring the stage by monitoring parasitic z motion of the stage.
As the stage moves over a sphere-cap there is parasitic motion in z during xy
movement. The highest z position is the centre of motion.
"""
@lt.thing_action @lt.thing_action
def recentre( def recentre(
self, self,
@ -22,7 +30,7 @@ class RecentringThing(lt.Thing):
max_steps=15, max_steps=15,
lateral_distance=5000, lateral_distance=5000,
): ):
"""Recentre the stage, based on the focal plane """Recentre the stage, based on the focal plane.
Autofocuses at multiple points around the sample to Autofocuses at multiple points around the sample to
find the overall maximum (or minimum) height, which find the overall maximum (or minimum) height, which
@ -43,7 +51,6 @@ class RecentringThing(lt.Thing):
much between these sites, making the procedure more sensitive much between these sites, making the procedure more sensitive
to noise or a failed autofocus. to noise or a failed autofocus.
""" """
max_steps = 20 max_steps = 20
dx = lateral_distance dx = lateral_distance

267
src/openflexure_microscope_server/things/autofocus.py
View file

@ -1,4 +1,4 @@
"""OpenFlexure Microscope autofocus module """OpenFlexure Microscope autofocus module.
This module defines a Thing that is responsible for using the stage and This module defines a Thing that is responsible for using the stage and
camera together to perform an autofocus routine. camera together to perform an autofocus routine.
@ -26,21 +26,7 @@ from .stage import StageDependency as Stage
class StackParams: class StackParams:
"""A class for holding for scan parameters """A class for holding for stack parameters, and returning computed ones."""
All arguments are keyword only
:param stack_dz: The number of motor steps between images
:param images_to_save: The number of images to save to disk
:param min_images_to_test: The minimum number of images in the stack before, the
stack is evaluated for focus. As more images are captured evaluation of the focus
is always evaluated with the same number of images. i.e. if min_images_to_test=9,
then 9 images are captured, if the stack is not well focused, a 10th image is
captured and images 2 to 10 are evaluated for focus
:param autofocus_dz: The number of steps in a full autofocus (when required)
:param images_dir: The directory to save images to disk
:param save_resolution: The resolution to save the captures to disk with
"""
def __init__( def __init__(
self, self,
@ -52,6 +38,20 @@ class StackParams:
images_dir: str, images_dir: str,
save_resolution: tuple[int, int], save_resolution: tuple[int, int],
) -> None: ) -> None:
"""Initialise the parameters. All arguments are keyword only.
:param stack_dz: The number of motor steps between images
:param images_to_save: The number of images to save to disk
:param min_images_to_test: The minimum number of images in the stack before,
the stack is evaluated for focus. As more images are captured evaluation
of the focus is always evaluated with the same number of images. i.e. if
``min_images_to_test=9``, then 9 images are captured, if the stack is not
well focused, a 10th image is captured and images 2 to 10 are evaluated
for focus
:param autofocus_dz: The number of steps in a full autofocus (when required)
:param images_dir: The directory to save images to disk
:param save_resolution: The resolution to save the captures to disk with
"""
if min_images_to_test < images_to_save: if min_images_to_test < images_to_save:
raise ValueError("Can't save more images than the minimum number tested") raise ValueError("Can't save more images than the minimum number tested")
if min_images_to_test % 2 == 0 or min_images_to_test <= 0: if min_images_to_test % 2 == 0 or min_images_to_test <= 0:
@ -97,19 +97,17 @@ class StackParams:
@property @property
def stack_z_range(self) -> int: def stack_z_range(self) -> int:
"""The range of the z stack, in steps """The range of the z stack, in steps.
Note that this is the range of the minimum number of images captured, Note that this is the range of the minimum number of images captured,
which is also the range of the images stored in memory that can be which is also the range of the images stored in memory that can be
saved.""" saved.
"""
return self.stack_dz * (self.min_images_to_test - 1) return self.stack_dz * (self.min_images_to_test - 1)
@property @property
def steps_undershoot(self) -> int: def steps_undershoot(self) -> int:
""" """The distance to deliberately undershoot the estimated optimal starting point."""
The distance to deliberately undershoot the estimated optimal starting point
"""
# Starting too low by "steps_undershoot" makes smart stacking faster. # Starting too low by "steps_undershoot" makes smart stacking faster.
# Starting a stack too high requires it to move to the start, # Starting a stack too high requires it to move to the start,
# autofocus and then re-stack. Starting slightly too low only # autofocus and then re-stack. Starting slightly too low only
@ -118,14 +116,14 @@ class StackParams:
@property @property
def max_images_to_test(self) -> int: def max_images_to_test(self) -> int:
"""The maximum number of images that will be captured and tested in a stack """The maximum number of images that will be captured and tested in a stack.
This is 15 images more then the minimum number that are captured. This is 15 images more then the minimum number that are captured.
""" """
return self.min_images_to_test + 15 return self.min_images_to_test + 15
def slice_to_save(self, sharpest_index): def slice_to_save(self, sharpest_index):
"""Return the slice of images to save given the index of the sharpest image""" """Return the slice of images to save given the index of the sharpest image."""
images_each_side = (self.images_to_save - 1) // 2 images_each_side = (self.images_to_save - 1) // 2
return slice( return slice(
max(sharpest_index - images_each_side, 0), max(sharpest_index - images_each_side, 0),
@ -135,9 +133,7 @@ class StackParams:
@dataclass @dataclass
class CaptureInfo: class CaptureInfo:
""" """The information from a capture in a z_stack."""
The information from a capture in a z_stack
"""
buffer_id: int buffer_id: int
position: dict[str, int] position: dict[str, int]
@ -145,7 +141,7 @@ class CaptureInfo:
@property @property
def filename(self) -> str: def filename(self) -> str:
"""The filename for this image generated from the position""" """The filename for this image generated from the position."""
return f"{self.position['x']}_{self.position['y']}_{self.position['z']}.jpeg" return f"{self.position['x']}_{self.position['y']}_{self.position['z']}.jpeg"
@ -157,21 +153,20 @@ def _get_capture_by_id(captures: list[CaptureInfo], buffer_id: int) -> CaptureIn
:returns: the CaptureInfo object of the capture with matching id :returns: the CaptureInfo object of the capture with matching id
:raises: ValueError if buffer_id does not match the buffer_id of any captures :raises ValueError: if buffer_id does not match the buffer_id of any captures
""" """
return captures[_get_capture_index_by_id(captures, buffer_id)] return captures[_get_capture_index_by_id(captures, buffer_id)]
def _get_capture_index_by_id(captures: list[CaptureInfo], buffer_id: int) -> int: def _get_capture_index_by_id(captures: list[CaptureInfo], buffer_id: int) -> int:
"""Return the index of the capture with the matching id from a list of CaptureInfo """Return the index of the capture with the matching id.
objects
:param captures: A list of capture objects :param captures: A list of capture objects
:param buffer_id: The buffer id of the image to return :param buffer_id: The buffer id of the image to return
:returns: the list index of the capture with matching id :returns: the list index of the capture with matching id
:raises: ValueError if buffer_id does not match the buffer_id of any captures :raises ValueError: if buffer_id does not match the buffer_id of any captures
""" """
ids = [capture.buffer_id for capture in captures] ids = [capture.buffer_id for capture in captures]
if buffer_id not in ids: if buffer_id not in ids:
@ -180,6 +175,13 @@ def _get_capture_index_by_id(captures: list[CaptureInfo], buffer_id: int) -> int
class SharpnessDataArrays(BaseModel): class SharpnessDataArrays(BaseModel):
"""A BaseModel with the position and sharpness data from JPEGSharpnessMonitor.
Each JPEG Size (representing a sharpness metric) has an associated timestamp,
as does each stage position. The stage positions need to be interpolated so
they correspond with the image timestamps.
"""
jpeg_times: NDArray jpeg_times: NDArray
jpeg_sizes: NDArray jpeg_sizes: NDArray
stage_times: NDArray stage_times: NDArray
@ -187,7 +189,29 @@ class SharpnessDataArrays(BaseModel):
class JPEGSharpnessMonitor: class JPEGSharpnessMonitor:
"""A class with direct access to the CameraThing for monitoring the MJPEG stream.
The autofocus algorithm uses sharpness calculated from the file size of the
images in the MJPEG stream. This class monitors both the stage position and the
jpeg sharpness over time.
The ``run`` context manager is used to start monitoring the camera stream. Position
monitoring happens during ``focus_rel``. Raw data can be retrieved with
``data_dict`` and data with interpolated ``z`` positions can be retrieved with
move_data.
A new JPEGSharpnessMonitor instance is created each time an action with the
SharpnessMonitorDep as an argument is called.
"""
def __init__(self, stage: Stage, camera: Camera, portal: lt.deps.BlockingPortal): def __init__(self, stage: Stage, camera: Camera, portal: lt.deps.BlockingPortal):
"""Initialise a new JPEGSharpnessMonitor. The args are injected automatically.
:param stage: A direct_thing_client dependency for the the microscope stage.
:param camera: A raw_thing_client depeendency for the camera. This is a raw
dependency as the underlying class needs to be
:param portal: The asyncio blocking portal for asynchronous task scheduling.
"""
self.camera = camera self.camera = camera
self.stage = stage self.stage = stage
self.portal = portal self.portal = portal
@ -200,7 +224,7 @@ class JPEGSharpnessMonitor:
running = False running = False
async def monitor_sharpness(self): async def monitor_sharpness(self):
"""Start monitoring the frame sizes""" """Start monitoring the frame sizes."""
self.running = True self.running = True
async for frame in self.camera.lores_mjpeg_stream.frame_async_generator(): async for frame in self.camera.lores_mjpeg_stream.frame_async_generator():
self.jpeg_times.append(time.time()) self.jpeg_times.append(time.time())
@ -210,7 +234,7 @@ class JPEGSharpnessMonitor:
@contextmanager @contextmanager
def run(self): def run(self):
"""Context manager, during which we will monitor sharpness from the camera""" """Context manager, during which we will monitor sharpness from the camera."""
self.portal.start_task_soon(self.monitor_sharpness) self.portal.start_task_soon(self.monitor_sharpness)
try: try:
yield yield
@ -218,6 +242,16 @@ class JPEGSharpnessMonitor:
self.running = False self.running = False
def focus_rel(self, dz: int, **kwargs) -> tuple[int, int]: def focus_rel(self, dz: int, **kwargs) -> tuple[int, int]:
"""Move the stage by dz, monitoring the position over time.
This performs exactly one move. Multiple calls of this method
will append to the internal position storage for more complex
autofocus procedures.
This should be run from within the JPEGSharpnessMonitor.run
context manager so that sharpness data and timestamps are also
collected.
"""
# Store the start time and position # Store the start time and position
self.stage_times.append(time.time()) self.stage_times.append(time.time())
self.stage_positions.append(self.stage.position) self.stage_positions.append(self.stage.position)
@ -238,7 +272,7 @@ class JPEGSharpnessMonitor:
def move_data( def move_data(
self, istart: int, istop: Optional[int] = None self, istart: int, istop: Optional[int] = None
) -> tuple[np.ndarray, np.ndarray, np.ndarray]: ) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
"""Extract sharpness as a function of (interpolated) z""" """Extract sharpness as a function of (interpolated) z."""
if istop is None: if istop is None:
istop = istart + 2 istop = istart + 2
jpeg_times: np.ndarray = np.array(self.jpeg_times) jpeg_times: np.ndarray = np.array(self.jpeg_times)
@ -266,7 +300,7 @@ class JPEGSharpnessMonitor:
return jpeg_times, jpeg_zs, jpeg_sizes[start:stop] return jpeg_times, jpeg_zs, jpeg_sizes[start:stop]
def sharpest_z_on_move(self, index: int) -> int: def sharpest_z_on_move(self, index: int) -> int:
"""Return the z position of the sharpest image on a given move""" """Return the z position of the sharpest image on a given move."""
_, jz, js = self.move_data(index) _, jz, js = self.move_data(index)
if len(js) == 0: if len(js) == 0:
raise ValueError( raise ValueError(
@ -275,7 +309,7 @@ class JPEGSharpnessMonitor:
return jz[np.argmax(js)] return jz[np.argmax(js)]
def data_dict(self) -> SharpnessDataArrays: def data_dict(self) -> SharpnessDataArrays:
"""Return the gathered data as a single convenient dictionary""" """Return the gathered data as a single convenient dictionary."""
data = {} data = {}
for k in ["jpeg_times", "jpeg_sizes", "stage_times", "stage_positions"]: for k in ["jpeg_times", "jpeg_sizes", "stage_times", "stage_positions"]:
data[k] = getattr(self, k) data[k] = getattr(self, k)
@ -290,7 +324,8 @@ class AutofocusThing(lt.Thing):
Actions here involve moving a stage in z, and using the camera to either Actions here involve moving a stage in z, and using the camera to either
capture images (generally, z-stacking) and measuring the sharpness of the capture images (generally, z-stacking) and measuring the sharpness of the
field of view to assess focus (autofocus and testing the success of a z-stack)""" field of view to assess focus (autofocus and testing the success of a z-stack)
"""
@lt.thing_action @lt.thing_action
def fast_autofocus( def fast_autofocus(
@ -299,7 +334,7 @@ class AutofocusThing(lt.Thing):
dz: int = 2000, dz: int = 2000,
start: str = "centre", start: str = "centre",
) -> SharpnessDataArrays: ) -> SharpnessDataArrays:
"""Sweep the stage up and down, then move to the sharpest point """Sweep the stage up and down, then move to the sharpest point.
This method will will move down by dz/2, sweep up by dz, and then evaluate This method will will move down by dz/2, sweep up by dz, and then evaluate
the position where the image was sharpest. We'll then move back down, and the position where the image was sharpest. We'll then move back down, and
@ -329,16 +364,16 @@ class AutofocusThing(lt.Thing):
dz: Sequence[int], dz: Sequence[int],
wait: float = 0, wait: float = 0,
) -> SharpnessDataArrays: ) -> SharpnessDataArrays:
"""Make a move (or a series of moves) and monitor sharpness """Make a move (or a series of moves) and monitor sharpness.
This method will will make a series of relative moves in z, and This method will will make a series of relative moves in z, and
return the sharpness (JPEG size) vs time, along with timestamps return the sharpness (JPEG size) vs time, along with timestamps
for the moves. This can be used to calibrate autofocus. for the moves. This can be used to calibrate autofocus.
Each move is relative to the last one, i.e. we will finish at Each move is relative to the last one, i.e. we will finish at
`sum(dz)` relative to the starting position. ``sum(dz)`` relative to the starting position.
If `wait` is specified, we will wait for that many seconds If ``wait`` is specified, we will wait for that many seconds
between moves. between moves.
""" """
with sharpness_monitor.run(): with sharpness_monitor.run():
@ -358,9 +393,9 @@ class AutofocusThing(lt.Thing):
): ):
"""Repeatedly autofocus the stage until it looks focused. """Repeatedly autofocus the stage until it looks focused.
This action will run the `fast_autofocus` action until it settles on a point This action will run the ``fast_autofocus`` action until it settles on a point
in the middle 3/5 of its range. Such logic can be helpful if the microscope in the middle 3/5 of its range. Such logic can be helpful if the microscope
is close to focus, but not quite within `dz/2`. It will attempt to autofocus is close to focus, but not quite within ``dz/2``. It will attempt to autofocus
up to 10 times. up to 10 times.
""" """
repeat = True repeat = True
@ -397,33 +432,34 @@ class AutofocusThing(lt.Thing):
stack_images_to_save = lt.ThingSetting( stack_images_to_save = lt.ThingSetting(
initial_value=1, initial_value=1,
model=int, model=int,
description="""The number of images to save in a stack.
Defaults to 1 unless you need to see either side of focus""",
) )
"""The number of images to save in a stack.
Defaults to 1 unless you need to see either side of focus
"""
stack_min_images_to_test = lt.ThingSetting( stack_min_images_to_test = lt.ThingSetting(
initial_value=9, initial_value=9,
model=int, model=int,
description="""The minimum number of images to capture in a stack.
This many images are captures and tested for focus, if the focus
is not central enough more images may be captured. After new images
are captured the number sets the number of images used for checking
if focus is central.
Defaults to 9 which balances reliability and speed
""",
) )
"""The minimum number of images to capture in a stack.
stack_dz = lt.ThingSetting( This many images are captures and tested for focus, if the focus is not central
initial_value=50, enough more images may be captured. After new images are captured the number sets
model=int, the number of images used for checking if focus is central.
description="""Space in steps between images in a z-stack
Suggested is 50 for 60-100x Defaults to 9 which balances reliability and speed/
100 for 40x """
200 for 20x""",
) stack_dz = lt.ThingSetting(initial_value=50, model=int)
"""Distance in steps between images in a z-stack.
Suggested values:
* 50 for 60-100x
* 100 for 40x
* 200 for 20x
"""
@lt.thing_action @lt.thing_action
def run_smart_stack( def run_smart_stack(
@ -435,26 +471,30 @@ class AutofocusThing(lt.Thing):
autofocus_dz: int, autofocus_dz: int,
save_resolution: tuple[int, int], save_resolution: tuple[int, int],
) -> tuple[bool, int]: ) -> tuple[bool, int]:
"""Run a smart stack, which captures images offset in z, testing """Run a smart stack.
whether the sharpest image is towards the centre of the stack.
The sharpest image, and optionally images around the sharpest, A smart stack captures images offset in z, testing whether the sharpest image
will be saved using their coordinates to images_dir is towards the centre of the stack.
The sharpest image, and optionally images around the sharpest, will be saved
to the images_dir with their coordinates in the filename.
:param cam: Camera Dependency supplied by LabThings dependency injection :param cam: Camera Dependency supplied by LabThings dependency injection
:param stage: Stage Dependency supplied by LabThings dependency injection :param stage: Stage Dependency supplied by LabThings dependency injection
:param sharpness_monitor: Sharpness Monitor Dependency (for focus detection) :param sharpness_monitor: Sharpness Monitor Dependency (for focus detection)
supplied by LabThings dependency injection supplied by LabThings dependency injection
:param images_dir: the folder to save all images :param images_dir: the folder to save all images
:param autofocus_dz: the range to autofocus over if a stack fails :param autofocus_dz: the range to autofocus over if a stack fails
:param save_resolution: The resolution the images should be saved at, the :param save_resolution: The resolution the images should be saved at, the
images will be resampled if this doesn't match the camera's capture resolution images will be resampled if this doesn't match the camera's capture
resolution
:returns: A tuple containing: :returns: A tuple containing:
- A boolean, True if stack was successfully
- The z position of the sharpest image
"""
* A boolean, True if stack was successfully
* The z position of the sharpest image
"""
# Set the variables to prevent changes from the GUI or other windows # Set the variables to prevent changes from the GUI or other windows
stack_parameters = StackParams( stack_parameters = StackParams(
stack_dz=self.stack_dz, stack_dz=self.stack_dz,
@ -465,9 +505,9 @@ class AutofocusThing(lt.Thing):
save_resolution=save_resolution, save_resolution=save_resolution,
) )
trys = 0 tries = 0
# Loop until a stack is successful # Loop until a stack is successful
while trys < stack_parameters.max_attempts: while tries < stack_parameters.max_attempts:
success, captures, sharpest_id = self.z_stack( success, captures, sharpest_id = self.z_stack(
stack_parameters=stack_parameters, stack_parameters=stack_parameters,
cam=cam, cam=cam,
@ -508,14 +548,13 @@ class AutofocusThing(lt.Thing):
stage: Stage, stage: Stage,
sharpness_monitor: SharpnessMonitorDep, sharpness_monitor: SharpnessMonitorDep,
) -> None: ) -> None:
"""Return to the initial height of the current stack, and run """Return to the initial z position and run a looping autofocus.
a looping autofocus.
Arguments: :param initial_z_pos: The initial z positions of previous captures
initial_z_pos: The initial z positions of previous captures :param autofocus_dz: the range in steps to autofocus
autofocus_dz: the range in steps to autofocus
variables stage and sharpness_monitor are Thing dependencies passed through from ``stage`` and ``sharpness_monitor`` are Thing dependencies passed through
the calling action from the calling action.
""" """
stage.move_absolute(z=initial_z_pos) stage.move_absolute(z=initial_z_pos)
self.looping_autofocus( self.looping_autofocus(
@ -531,17 +570,18 @@ class AutofocusThing(lt.Thing):
stack_parameters: StackParams, stack_parameters: StackParams,
cam: WrappedCamera, cam: WrappedCamera,
) -> int: ) -> int:
"""Save the required captures to disk. Will save the sharpest image, """Save the required captures to disk.
and any images either side of focus.
This will save the sharpest image, and optionally extra images either
side of focus (see ``stack_parameters.images_to_save``).
:param sharpest_id: the buffer id index of the sharpest image
:param captures: a list of captures, including file name, image data and
metadata
:param stack_parameters: a StackParams object holding stack parameters
:param cam: is a Thing dependency passed through from the calling action
Arguments:
sharpest_id: the buffer id index of the sharpest image
captures: a list of captures, including file name, image data and metadata
stack_parameters: a StackParams object holding stack parameters
variables logger and capture are Thing dependencies passed through from the
calling action
""" """
sharpest_index = _get_capture_index_by_id(captures, sharpest_id) sharpest_index = _get_capture_index_by_id(captures, sharpest_id)
slice_to_save = stack_parameters.slice_to_save(sharpest_index) slice_to_save = stack_parameters.slice_to_save(sharpest_index)
@ -561,17 +601,22 @@ class AutofocusThing(lt.Thing):
cam: WrappedCamera, cam: WrappedCamera,
stage: Stage, stage: Stage,
) -> tuple[bool, list[CaptureInfo], Optional[int]]: ) -> tuple[bool, list[CaptureInfo], Optional[int]]:
"""Capture a series of images offset by stack_parameters.stack_dz, and test whether """Capture a series of images checking that sharpest image central.
the sharpest image is towards the centre of the stack.
The images are separated in z offset by stack_parameters.stack_dz, as they
are captured the last stack_parameters.min_images_to_test images are checked
to see if the sharpest image is central enough in the stack. If it is the stack
completes.
:param stack_parameters: a StackParams object holding stack parameters :param stack_parameters: a StackParams object holding stack parameters
:param cam: Camera Dependency to be passed through from the calling action :param cam: Camera Dependency to be passed through from the calling action
:param stage: Stage Dependency to be passed through from the calling action :param stage: Stage Dependency to be passed through from the calling action
:returns: A tuple of :returns: A tuple of
- the stack result (True for successful stack, False for failed stack),
- a list of CaptureInfo objects, * the stack result (True for successful stack, False for failed stack),
- the buffer_id of the shapest image (or None if the stack failed) * a list of CaptureInfo objects,
* the buffer_id of the sharpest image (or None if the stack failed).
""" """
# Move down by the height of the z stack, plus an overshoot # Move down by the height of the z stack, plus an overshoot
# Better to start too low and take too many images than too high and need to refocus # Better to start too low and take too many images than too high and need to refocus
@ -628,11 +673,11 @@ class AutofocusThing(lt.Thing):
:param cam: Camera Dependency to be passed through from the calling action :param cam: Camera Dependency to be passed through from the calling action
:param stage: Stage Dependency to be passed through from the calling action :param stage: Stage Dependency to be passed through from the calling action
:buffer_max: The maximum number of images to tell the camera to keep in memory :param buffer_max: The maximum number of images to tell the camera to keep in memory
for saving once the stack is complete for saving once the stack is complete
:return: A CaptureInfo object containing the capture information including its :returns: A CaptureInfo object containing the capture information including its
camera buffer_id needed for saving. camera buffer_id needed for saving.
""" """
stage_location = stage.position stage_location = stage.position
buffer_id = cam.capture_to_memory(buffer_max=buffer_max) buffer_id = cam.capture_to_memory(buffer_max=buffer_max)
@ -645,18 +690,22 @@ class AutofocusThing(lt.Thing):
def check_stack_result( def check_stack_result(
self, captures: list[CaptureInfo] self, captures: list[CaptureInfo]
) -> tuple[Literal["success", "continue", "restart"], int]: ) -> tuple[Literal["success", "continue", "restart"], int]:
"""Test a list of captures, to decide whether the sharpest image from a """Check if the sharpest image in a list of captures is central enough.
stack is centrally enough in the stack
:param captures: a list of the capture objects to for testing if the :param captures: a list of the capture objects to for testing if the
sharpeness has converged in the centre sharpness has converged in the centre
:return: A tuple with two values: :returns: A tuple with two values:
- result - which is one of three literal values:
'success' if the sharpest image is towards the centre * result - which is one of three literal values:
'continue' if the sharpest image is in the final two images of the list
'restart' if the sharpest image is in the first two images of the list * ``success`` if the sharpest image is towards the centre
- capture_id - the buffer id of the sharpest image * ``continue`` if the sharpest image is in the final two images of the
list
* ``restart`` if the sharpest image is in the first two images of the
list
* capture_id - the buffer id of the sharpest image
""" """
sharpest_index = np.argmax([capture.sharpness for capture in captures]) sharpest_index = np.argmax([capture.sharpness for capture in captures])
# The buffer id of the sharpest image # The buffer id of the sharpest image

36
src/openflexure_microscope_server/things/background_detect.py
View file

@ -1,3 +1,10 @@
"""Provide functionality to detect if the camera is imaging sample or background.
An example background image must be captured and analysed by BackgroundDetectThing,
information from this images is used to detect whether the current camera field of
view contains sample.
"""
from typing import Mapping, Optional from typing import Mapping, Optional
import cv2 import cv2
import numpy as np import numpy as np
@ -10,19 +17,31 @@ from .camera import CameraDependency as CamDep
class ChannelDistributions(BaseModel): class ChannelDistributions(BaseModel):
"""A BaseModel for storing the channel distribution of a background image."""
means: list[float] means: list[float]
"""The mean of each channel in the colourspace."""
standard_deviations: list[float] standard_deviations: list[float]
"""The standard deviation of each channel in the colourspace."""
colorspace: str = "LUV" colorspace: str = "LUV"
"""The colourspace used."""
class BackgroundDetectThing(lt.Thing): class BackgroundDetectThing(lt.Thing):
"""Thing for setting a background image and detecting sample in the field of view.
This uses an LUV colour space checking only the mean and standard deviation of the
UV channels. Over time different, selectable, background detection methods will be
added.
"""
# Requires a getter and a setter to support being a BaseModel but being # Requires a getter and a setter to support being a BaseModel but being
# saved to file as a dict # saved to file as a dict
_background_distributions: Optional[ChannelDistributions] = None _background_distributions: Optional[ChannelDistributions] = None
@lt.thing_setting @lt.thing_setting
def background_distributions(self) -> Optional[ChannelDistributions]: def background_distributions(self) -> Optional[ChannelDistributions]:
"""The statistics of the background image""" """The statistics of the background image."""
bd = self._background_distributions bd = self._background_distributions
if bd is None: if bd is None:
return None return None
@ -46,19 +65,19 @@ class BackgroundDetectThing(lt.Thing):
tolerance = lt.ThingSetting( tolerance = lt.ThingSetting(
initial_value=7.0, initial_value=7.0,
model=float, model=float,
description="How many standard deviations to allow for the background",
) )
"""How many standard deviations to allow for the background."""
fraction = lt.ThingSetting( fraction = lt.ThingSetting(
initial_value=25.0, initial_value=25.0,
model=float, model=float,
description="How much of the image needs to be not background to label as sample",
) )
"""How much of the image needs to be not background to label as sample"""
def background_mask(self, image: np.ndarray) -> np.ndarray: def background_mask(self, image: np.ndarray) -> np.ndarray:
"""Calculate a binary image, showing whether each pixel is background """Calculate a binary image, showing whether each pixel is background.
The image should be in LUV format, the ouput will be binary with the The image should be in LUV format, the output will be binary with the
same shape in the first two dimensions. same shape in the first two dimensions.
""" """
d = self.background_distributions d = self.background_distributions
@ -78,7 +97,7 @@ class BackgroundDetectThing(lt.Thing):
@lt.thing_action @lt.thing_action
def background_fraction(self, cam: CamDep) -> float: def background_fraction(self, cam: CamDep) -> float:
"""Determine what fraction of the current image is background """Determine what fraction of the current image is background.
This action will acquire a new image from the preview stream, then This action will acquire a new image from the preview stream, then
evaluate whether it is foreground or background, by comparing it evaluate whether it is foreground or background, by comparing it
@ -96,7 +115,7 @@ class BackgroundDetectThing(lt.Thing):
@lt.thing_action @lt.thing_action
def image_is_sample(self, cam: CamDep) -> bool: def image_is_sample(self, cam: CamDep) -> bool:
"""Label the current image as either background or sample""" """Label the current image as either background or sample."""
b_fraction = self.background_fraction(cam) b_fraction = self.background_fraction(cam)
fraction_threshold = self.fraction fraction_threshold = self.fraction
@ -104,7 +123,7 @@ class BackgroundDetectThing(lt.Thing):
@lt.thing_action @lt.thing_action
def set_background(self, cam: CamDep): def set_background(self, cam: CamDep):
"""Grab an image, and use its statistics to set the background """Grab an image, and use its statistics to set the background.
This should be run when the microscope is looking at an empty region, This should be run when the microscope is looking at an empty region,
and will calculate the mean and standard deviation of the pixel values and will calculate the mean and standard deviation of the pixel values
@ -135,6 +154,7 @@ class BackgroundDetectThing(lt.Thing):
@property @property
def thing_state(self) -> Mapping: def thing_state(self) -> Mapping:
"""Summary metadata describing the current state of the Thing."""
bd = self.background_distributions bd = self.background_distributions
return { return {
"background_distributions": bd.model_dump() if bd else None, "background_distributions": bd.model_dump() if bd else None,

130
src/openflexure_microscope_server/things/camera/__init__.py
View file

@ -1,7 +1,7 @@
"""OpenFlexure Microscope Camera """OpenFlexure Microscope Camera.
This module defines the interface for cameras. Any compatible lt.Thing This module defines the interface for cameras. Any compatible lt.Thing
should enabe the server to work. should enable the server to work.
See repository root for licensing information. See repository root for licensing information.
""" """
@ -19,39 +19,41 @@ from labthings_fastapi.types.numpy import NDArray
class JPEGBlob(lt.blob.Blob): class JPEGBlob(lt.blob.Blob):
"""A class representing a JPEG image as a LabThings FastAPI Blob."""
media_type: str = "image/jpeg" media_type: str = "image/jpeg"
class PNGBlob(lt.blob.Blob): class PNGBlob(lt.blob.Blob):
"""A class representing a PNG image as a LabThings FastAPI Blob""" """A class representing a PNG image as a LabThings FastAPI Blob."""
media_type: str = "image/png" media_type: str = "image/png"
class ArrayModel(RootModel): class ArrayModel(RootModel):
"""A model for an array""" """A model for an array."""
root: NDArray root: NDArray
class CaptureError(RuntimeError): class CaptureError(RuntimeError):
"""An error trying to capture from a CameraThing""" """An error trying to capture from a CameraThing."""
class NoImageInMemoryError(RuntimeError): class NoImageInMemoryError(RuntimeError):
"""An error called if no image in in memory when an method is called to use that image""" """An error called if no image is in memory when accessed."""
class CameraMemoryBuffer: class CameraMemoryBuffer:
""" """A class that holds images in memory. The images are by default PIL images.
A class that holds images in memory. The images are by default PIL images.
However subclasses of BaseCamera can use this class to store other object types However subclasses of BaseCamera can use this class to store other object types.
""" """
_storage: dict[int, tuple[Any, Optional[dict]]] _storage: dict[int, tuple[Any, Optional[dict]]]
def __init__(self): def __init__(self):
"""Create the buffer instance."""
# This dictionary is the main store for data. Dictionaries are ordered since # This dictionary is the main store for data. Dictionaries are ordered since
# Python 3.6, so the order in the dictionary is the capture order # Python 3.6, so the order in the dictionary is the capture order
self._storage = {} self._storage = {}
@ -62,20 +64,19 @@ class CameraMemoryBuffer:
def add_image( def add_image(
self, image: Any, metadata: Optional[dict] = None, buffer_max: int = 1 self, image: Any, metadata: Optional[dict] = None, buffer_max: int = 1
) -> int: ) -> int:
""" """Add an image to the Memory buffer.
Add an image to the Memory buffer
This will add an image to the memory buffer. By default the buffer will This will add an image to the memory buffer. By default the buffer will
be cleared. To allow saving multiple images the buffer_max must be set be cleared. To allow saving multiple images the buffer_max must be set
every time an image is added. every time an image is added.
:param image: The image to add. A PIL image is recommended, but cameras :param image: The image to add. A PIL image is recommended, but cameras
can choose to use other formats can choose to use other formats
:param metadata: Optional, a dictionary of the image metadata. :param metadata: Optional, a dictionary of the image metadata.
:param buffer_max: The maximum number of images that should be in the buffer :param buffer_max: The maximum number of images that should be in the buffer
once this images is added. Default is 1. once this images is added. Default is 1.
:return buffer_id: The id in the buffer for this image :returns: The id in the buffer for this image
""" """
self._latest_id += 1 self._latest_id += 1
self._create_space(buffer_max) self._create_space(buffer_max)
@ -85,8 +86,7 @@ class CameraMemoryBuffer:
def get_image( def get_image(
self, buffer_id: Optional[int] = None, remove: bool = True self, buffer_id: Optional[int] = None, remove: bool = True
) -> tuple[Any, Optional[dict]]: ) -> tuple[Any, Optional[dict]]:
""" """Return the image with the given id.
Return the image with the given id.
If no id is given the most recent image is returned. However, the If no id is given the most recent image is returned. However, the
buffer is also cleared, otherwise it would be possible to accidentally buffer is also cleared, otherwise it would be possible to accidentally
@ -94,9 +94,8 @@ class CameraMemoryBuffer:
:param buffer_id: The buffer id of the image to retrieve :param buffer_id: The buffer id of the image to retrieve
:param remove: True (default) to remove this image from the buffer, False :param remove: True (default) to remove this image from the buffer, False
to leave the image in the buffer. to leave the image in the buffer.
""" """
# No id given # No id given
if buffer_id is None: if buffer_id is None:
# Get the latest image and metadata tuple from storage # Get the latest image and metadata tuple from storage
@ -118,17 +117,14 @@ class CameraMemoryBuffer:
) from e ) from e
def clear(self): def clear(self):
""" """Clear all images from memory."""
Clear all images from memory
"""
self._storage.clear() self._storage.clear()
def _create_space(self, buffer_max: int) -> None: def _create_space(self, buffer_max: int) -> None:
""" """Create space to add an image.
Create space to add an image.
:param buffer_max: The maximum number of images that should be in the buffer :param buffer_max: The maximum number of images that should be in the buffer
once another images is added. once another images is added.
""" """
# If only one image to be stored just clear the storage and return # If only one image to be stored just clear the storage and return
if buffer_max <= 1: if buffer_max <= 1:
@ -147,37 +143,45 @@ class CameraMemoryBuffer:
class BaseCamera(lt.Thing): class BaseCamera(lt.Thing):
"""The base class for all cameras. All cameras must directly inherit from this class""" """The base class for all cameras. All cameras must directly inherit from this class.
The connection to the camera hardware should be added to the ``__enter__`` method not
``__init__`` method of the subclass.
"""
mjpeg_stream = lt.outputs.MJPEGStreamDescriptor() mjpeg_stream = lt.outputs.MJPEGStreamDescriptor()
lores_mjpeg_stream = lt.outputs.MJPEGStreamDescriptor() lores_mjpeg_stream = lt.outputs.MJPEGStreamDescriptor()
_memory_buffer = CameraMemoryBuffer() _memory_buffer = CameraMemoryBuffer()
def __enter__(self) -> None: def __enter__(self) -> None:
"""Open hardware connection when the Thing context manager is opened."""
raise NotImplementedError("CameraThings must define their own __enter__ method") raise NotImplementedError("CameraThings must define their own __enter__ method")
def __exit__(self, _exc_type, _exc_value, _traceback) -> None: def __exit__(self, _exc_type, _exc_value, _traceback) -> None:
"""Close hardware connection when the Thing context manager is closed."""
raise NotImplementedError("CameraThings must define their own __exit__ method") raise NotImplementedError("CameraThings must define their own __exit__ method")
@lt.thing_action @lt.thing_action
def start_streaming( def start_streaming(
self, main_resolution: tuple[int, int], buffer_count: int self, main_resolution: tuple[int, int], buffer_count: int
) -> None: ) -> None:
"""Start (or stop and restart) the camera with the given resolution """Start (or stop and restart) the camera.
for the main stream, and buffer_count number of images in the buffer"""
:param main_resolution: the resolution to use for the main stream.
:param buffer_count: number of images in the stream buffer.
"""
raise NotImplementedError( raise NotImplementedError(
"CameraThings must define their own start_streaming method" "CameraThings must define their own start_streaming method"
) )
def kill_mjpeg_streams(self): def kill_mjpeg_streams(self):
""" """Kill the streams now as the server is shutting down.
Kill the streams now as the server is shutting down.
This is called when uvicorn gets the a shutdown signal. As this is called from This is called when uvicorn gets the a shutdown signal. As this is called from
the event loop it cannot interact with the our ThingProperties or run the event loop it cannot interact with the our ThingProperties or run
`self.mjpeg_stream.stop()` as the portal cannot be called from this loop. ``self.mjpeg_stream.stop()`` as the portal cannot be called from this loop.
Instead we just set the `_streaming` value to False. This stops the async frame Instead we just set the ``_streaming`` value to False. This stops the async frame
generator when the next frame notifies. generator when the next frame notifies.
""" """
if self.stream_active: if self.stream_active:
@ -186,7 +190,7 @@ class BaseCamera(lt.Thing):
@lt.thing_property @lt.thing_property
def stream_active(self) -> bool: def stream_active(self) -> bool:
"Whether the MJPEG stream is active" """Whether the MJPEG stream is active."""
raise NotImplementedError( raise NotImplementedError(
"CameraThings must define their own stream_active method" "CameraThings must define their own stream_active method"
) )
@ -197,6 +201,7 @@ class BaseCamera(lt.Thing):
stream_name: Literal["main", "lores", "raw", "full"] = "main", stream_name: Literal["main", "lores", "raw", "full"] = "main",
wait: Optional[float] = 5, wait: Optional[float] = 5,
) -> NDArray: ) -> NDArray:
"""Acquire one image from the camera and return as an array."""
raise NotImplementedError( raise NotImplementedError(
"CameraThings must define their own capture_array method" "CameraThings must define their own capture_array method"
) )
@ -208,7 +213,7 @@ class BaseCamera(lt.Thing):
resolution: Literal["lores", "main", "full"] = "main", resolution: Literal["lores", "main", "full"] = "main",
wait: Optional[float] = 5, wait: Optional[float] = 5,
) -> JPEGBlob: ) -> JPEGBlob:
"""Acquire one image from the camera and return as a JPEG blob""" """Acquire one image from the camera and return as a JPEG blob."""
raise NotImplementedError( raise NotImplementedError(
"CameraThings must define their own capture_jpeg method" "CameraThings must define their own capture_jpeg method"
) )
@ -219,9 +224,9 @@ class BaseCamera(lt.Thing):
portal: lt.deps.BlockingPortal, portal: lt.deps.BlockingPortal,
stream_name: Literal["main", "lores"] = "main", stream_name: Literal["main", "lores"] = "main",
) -> JPEGBlob: ) -> JPEGBlob:
"""Acquire one image from the preview stream and return as an array """Acquire one image from the preview stream and return as an array.
This differs from `capture_jpeg` in that it does not pause the MJPEG This differs from ``capture_jpeg`` in that it does not pause the MJPEG
preview stream. Instead, we simply return the next frame from that preview stream. Instead, we simply return the next frame from that
stream (either "main" for the preview stream, or "lores" for the low stream (either "main" for the preview stream, or "lores" for the low
resolution preview). No metadata is returned. resolution preview). No metadata is returned.
@ -238,7 +243,7 @@ class BaseCamera(lt.Thing):
portal: lt.deps.BlockingPortal, portal: lt.deps.BlockingPortal,
stream_name: Literal["main", "lores"] = "main", stream_name: Literal["main", "lores"] = "main",
) -> int: ) -> int:
"""Acquire one image from the preview stream and return its size""" """Acquire one image from the preview stream and return its size."""
stream = ( stream = (
self.lores_mjpeg_stream if stream_name == "lores" else self.mjpeg_stream self.lores_mjpeg_stream if stream_name == "lores" else self.mjpeg_stream
) )
@ -250,7 +255,7 @@ class BaseCamera(lt.Thing):
stream_name: Literal["main", "lores", "raw"], stream_name: Literal["main", "lores", "raw"],
wait: Optional[float], wait: Optional[float],
) -> None: ) -> None:
"""Capture a PIL image from stream stream_name with timeout wait""" """Capture a PIL image from stream stream_name with timeout wait."""
raise NotImplementedError( raise NotImplementedError(
"CameraThings must define their own capture_image method" "CameraThings must define their own capture_image method"
) )
@ -263,15 +268,15 @@ class BaseCamera(lt.Thing):
metadata_getter: lt.deps.GetThingStates, metadata_getter: lt.deps.GetThingStates,
save_resolution: Optional[Tuple[int, int]] = None, save_resolution: Optional[Tuple[int, int]] = None,
) -> None: ) -> None:
"""Capture an image and save it to disk """Capture an image and save it to disk.
:param jpeg_path: The path to save the file to :param jpeg_path: The path to save the file to
:param logger: This should be injected automatically by Labthings FastAPI :param logger: This should be injected automatically by Labthings FastAPI
when calling the action when calling the action
:param metadata_getter: This should be injected automatically by Labthings :param metadata_getter: This should be injected automatically by Labthings
FastAPI when calling the action FastAPI when calling the action
:param save_resolution: can be set to resize the image before saving. By :param save_resolution: can be set to resize the image before saving. By
default this is None meaning that the image is saved at original resolution. default this is None meaning that the image is saved at original resolution.
""" """
image, metadata = self._robust_image_capture( image, metadata = self._robust_image_capture(
metadata_getter, metadata_getter,
@ -293,20 +298,19 @@ class BaseCamera(lt.Thing):
metadata_getter: lt.deps.GetThingStates, metadata_getter: lt.deps.GetThingStates,
buffer_max: int = 1, buffer_max: int = 1,
) -> None: ) -> None:
""" """Capture an image to memory. This can be saved later with ``save_from_memory``.
Capture an image to memory. This can be saved later with `save_from_memory`
Note that only one image is held in memory so this will overwrite any image Note that only one image is held in memory so this will overwrite any image
in memory. in memory.
:param logger: This should be injected automatically by Labthings FastAPI :param logger: This should be injected automatically by Labthings FastAPI
when calling the action when calling the action
:param metadata_getter: This should be injected automatically by Labthings :param metadata_getter: This should be injected automatically by Labthings
FastAPI when calling the action FastAPI when calling the action
:param buffer_max: The maximum number of images that should be in the buffer :param buffer_max: The maximum number of images that should be in the buffer
once this images is added. Default is 1. once this images is added. Default is 1.
:return: the buffer id of the image captured :returns: the buffer id of the image captured
""" """
image, metadata = self._robust_image_capture(metadata_getter, logger) image, metadata = self._robust_image_capture(metadata_getter, logger)
return self._memory_buffer.add_image(image, metadata, buffer_max=buffer_max) return self._memory_buffer.add_image(image, metadata, buffer_max=buffer_max)
@ -319,16 +323,16 @@ class BaseCamera(lt.Thing):
save_resolution: Optional[Tuple[int, int]] = None, save_resolution: Optional[Tuple[int, int]] = None,
buffer_id: Optional[int] = None, buffer_id: Optional[int] = None,
) -> None: ) -> None:
""" """Save an image that has been captured to memory.
Save an image that has been captured to memory.
:param jpeg_path: The path to save the file to :param jpeg_path: The path to save the file to
:param logger: This should be injected automatically by Labthings FastAPI :param logger: This should be injected automatically by Labthings FastAPI
when calling the action when calling the action
:param save_resolution: can be set to resize the image before saving. By :param save_resolution: can be set to resize the image before saving. By
default this is None meaning that the image is saved at original resolution. default this is None meaning that the image is saved at original
resolution.
:param buffer_id: The buffer id of the image to save, this was returned by :param buffer_id: The buffer id of the image to save, this was returned by
`capture_to_memory` ``capture_to_memory``
""" """
image, metadata = self._memory_buffer.get_image(buffer_id) image, metadata = self._memory_buffer.get_image(buffer_id)
@ -342,7 +346,7 @@ class BaseCamera(lt.Thing):
@lt.thing_action @lt.thing_action
def clear_buffers(self) -> None: def clear_buffers(self) -> None:
"""Clear all images in memory""" """Clear all images in memory."""
self._memory_buffer.clear() self._memory_buffer.clear()
def _robust_image_capture( def _robust_image_capture(
@ -350,12 +354,14 @@ class BaseCamera(lt.Thing):
metadata_getter: lt.deps.GetThingStates, metadata_getter: lt.deps.GetThingStates,
logger: lt.deps.InvocationLogger, logger: lt.deps.InvocationLogger,
) -> Image: ) -> Image:
"""Capture an image in memory and return it with metadata """Capture an image in memory and return it with metadata.
CaptureError raised if the capture fails for any reason
returns tuple with PIL Image, and dict of metadata.
This robust capturing method attempts to capture the image five times This robust capturing method attempts to capture the image five times
each time with a 5 second timeout set. each time with a 5 second timeout set.
:raises CaptureError: if the capture fails for any reason
:returns: tuple with PIL Image, and dictionary of metadata.
""" """
for capture_attempts in range(5): for capture_attempts in range(5):
try: try:
@ -376,10 +382,14 @@ class BaseCamera(lt.Thing):
logger: lt.deps.InvocationLogger, logger: lt.deps.InvocationLogger,
save_resolution: Optional[Tuple[int, int]] = None, save_resolution: Optional[Tuple[int, int]] = None,
) -> None: ) -> None:
"""Saving the captured image and metadata to disk """Save the captured image and metadata to disk.
logger warning (via InvocationLogger) is raised if metadata is failed to be added
IOError is raised if the file cannot be saved A warning (via InvocationLogger) is raised if metadata is failed to be added
nothing is returned on success"""
:raises IOError: if the file cannot be saved
nothing is returned on success
"""
if save_resolution is not None and image.size != save_resolution: if save_resolution is not None and image.size != save_resolution:
image = image.resize(save_resolution, Image.BOX) image = image.resize(save_resolution, Image.BOX)
try: try:

25
src/openflexure_microscope_server/things/camera/opencv.py
View file

@ -1,7 +1,7 @@
"""OpenFlexure Microscope OpenCV Camera """OpenFlexure Microscope OpenCV Camera.
This module defines a camera Thing that uses OpenCV's This module defines a camera Thing that uses OpenCV's
`VideoCapture`. ``VideoCapture``.
See repository root for licensing information. See repository root for licensing information.
""" """
@ -23,14 +23,19 @@ from . import BaseCamera, JPEGBlob
class OpenCVCamera(BaseCamera): class OpenCVCamera(BaseCamera):
"""A Thing representing an OpenCV camera""" """A Thing that provides and interface to an OpenCV Camera."""
def __init__(self, camera_index: int = 0): def __init__(self, camera_index: int = 0):
"""Iniatilise the thing storing the index of the camera to use.
:param camera_index: The index of the camera to use for the microscope.
"""
self.camera_index = camera_index self.camera_index = camera_index
self._capture_thread: Optional[Thread] = None self._capture_thread: Optional[Thread] = None
self._capture_enabled = False self._capture_enabled = False
def __enter__(self): def __enter__(self):
"""Start the capture thread when the Thing context manager is opened."""
self.cap = cv2.VideoCapture(self.camera_index) self.cap = cv2.VideoCapture(self.camera_index)
self._capture_enabled = True self._capture_enabled = True
self._capture_thread = Thread(target=self._capture_frames) self._capture_thread = Thread(target=self._capture_frames)
@ -38,6 +43,10 @@ class OpenCVCamera(BaseCamera):
return self return self
def __exit__(self, _exc_type, _exc_value, _traceback): def __exit__(self, _exc_type, _exc_value, _traceback):
"""Release the camera when the Thing context manager is closed.
Before releasing the camera the capture thread is closed.
"""
if self.stream_active: if self.stream_active:
self._capture_enabled = False self._capture_enabled = False
self._capture_thread.join() self._capture_thread.join()
@ -45,7 +54,7 @@ class OpenCVCamera(BaseCamera):
@lt.thing_property @lt.thing_property
def stream_active(self) -> bool: def stream_active(self) -> bool:
"Whether the MJPEG stream is active" """Whether the MJPEG stream is active."""
if self._capture_enabled and self._capture_thread: if self._capture_enabled and self._capture_thread:
return self._capture_thread.is_alive() return self._capture_thread.is_alive()
return False return False
@ -71,13 +80,13 @@ class OpenCVCamera(BaseCamera):
self, self,
resolution: Literal["main", "full"] = "full", resolution: Literal["main", "full"] = "full",
) -> NDArray: ) -> NDArray:
"""Acquire one image from the camera and return as an array """Acquire one image from the camera and return as an array.
This function will produce a nested list containing an uncompressed RGB image. This function will produce a nested list containing an uncompressed RGB image.
It's likely to be highly inefficient - raw and/or uncompressed captures using It's likely to be highly inefficient - raw and/or uncompressed captures using
binary image formats will be added in due course. binary image formats will be added in due course.
""" """
logging.warning(f"OpenCV camera doen't respect {resolution} setting") logging.warning(f"OpenCV camera doesn't respect {resolution} setting")
ret, frame = self.cap.read() ret, frame = self.cap.read()
if not ret: if not ret:
raise RuntimeError( raise RuntimeError(
@ -91,11 +100,11 @@ class OpenCVCamera(BaseCamera):
metadata_getter: lt.deps.GetThingStates, metadata_getter: lt.deps.GetThingStates,
resolution: Literal["main", "full"] = "main", resolution: Literal["main", "full"] = "main",
) -> JPEGBlob: ) -> JPEGBlob:
"""Acquire one image from the camera and return as a JPEG blob """Acquire one image from the camera and return as a JPEG blob.
This function will produce a JPEG image. This function will produce a JPEG image.
""" """
logging.warning(f"OpenCV camera doen't respect {resolution} setting") logging.warning(f"OpenCV camera doesn't respect {resolution} setting")
frame = self.capture_array() frame = self.capture_array()
jpeg = cv2.imencode(".jpg", frame)[1].tobytes() jpeg = cv2.imencode(".jpg", frame)[1].tobytes()
exif_dict = { exif_dict = {

177
src/openflexure_microscope_server/things/camera/picamera.py
View file

@ -1,5 +1,4 @@
""" """Submodule for interacting with a Raspberry Pi camera using the Picamera2 library.
This SubModule interacts with a Raspberry Pi camera using the Picamera2 library.
The Picamera2 library uses LibCamera as the underlying camera stack. This gives us The Picamera2 library uses LibCamera as the underlying camera stack. This gives us
some control of the GPU pipeline for the image. some control of the GPU pipeline for the image.
@ -41,10 +40,10 @@ from . import BaseCamera, JPEGBlob, ArrayModel
class PicameraStreamOutput(Output): class PicameraStreamOutput(Output):
"""An Output class that sends frames to a stream""" """An Output class that sends frames to a stream."""
def __init__(self, stream: lt.outputs.MJPEGStream, portal: lt.deps.BlockingPortal): def __init__(self, stream: lt.outputs.MJPEGStream, portal: lt.deps.BlockingPortal):
"""Create an output that puts frames in an MJPEGStream """Create an output that puts frames in an MJPEGStream.
We need to pass the stream object, and also the blocking portal, because We need to pass the stream object, and also the blocking portal, because
new frame notifications happen in the anyio event loop and frames are new frame notifications happen in the anyio event loop and frames are
@ -58,14 +57,14 @@ class PicameraStreamOutput(Output):
def outputframe( def outputframe(
self, frame, _keyframe=True, _timestamp=None, _packet=None, _audio=False self, frame, _keyframe=True, _timestamp=None, _packet=None, _audio=False
): ):
"""Add a frame to the stream's ringbuffer""" """Add a frame to the stream's ringbuffer."""
self.stream.add_frame(frame, self.portal) self.stream.add_frame(frame, self.portal)
class SensorMode(BaseModel): class SensorMode(BaseModel):
""" """A Pydantic model holding all the information about a specific sensor mode.
A Pydantic model holding all the information about a specific
sensor mode as reported by the PiCamera. This data is as reported by the PiCamera2 module.
""" """
unpacked: str unpacked: str
@ -78,11 +77,11 @@ class SensorMode(BaseModel):
class SensorModeSelector(BaseModel): class SensorModeSelector(BaseModel):
""" """A Pydantic model holding the two values needed to select a PiCamera Sensor mode.
A Pydantic model holding the two values needed to select a PiCamera
Sensor mode. The output size and the bit depth.
This is a Pydantic modell so that it can be saved to the disk. These values are the output size and the bit depth.
This is a Pydantic model so that it can be saved to disk.
""" """
output_size: tuple[int, int] output_size: tuple[int, int]
@ -90,14 +89,13 @@ class SensorModeSelector(BaseModel):
class LensShading(BaseModel): class LensShading(BaseModel):
""" """A Pydantic model holding the lens shading tables.
A Pydantic model holding the lens shading tables.
PiCamera needs three numpy arrays for lens shading correction. Each array is PiCamera needs three numpy arrays for lens shading correction. Each array is
(12, 16) in size. The arrays are luminance, red-difference chroma (Cr), and (12, 16) in size. The arrays are luminance, red-difference chroma (Cr), and
blue-difference chroma (Cb). blue-difference chroma (Cb).
This is a Pydantic modell so that it can be saved to the disk. This is a Pydantic model so that it can be saved to the disk.
""" """
luminance: list[list[float]] luminance: list[list[float]]
@ -106,9 +104,20 @@ class LensShading(BaseModel):
class StreamingPiCamera2(BaseCamera): class StreamingPiCamera2(BaseCamera):
"""A Thing that represents an OpenCV camera""" """A Thing that provides and interface to the Raspberry Pi Camera.
Currently the Thing only supports the PiCamera v2 board. This needs
generalisation.
"""
def __init__(self, camera_num: int = 0): def __init__(self, camera_num: int = 0):
"""Initialise the camera with the given camera number.
This makes no connection to the camera (except to get the default tuning file).
:param camera_num: The number of the camera. This should generally be left as 0
as most Raspberry Pi boards only support 1 camera.
"""
self._setting_save_in_progress = False self._setting_save_in_progress = False
self.camera_num = camera_num self.camera_num = camera_num
self.camera_configs: dict[str, dict] = {} self.camera_configs: dict[str, dict] = {}
@ -130,22 +139,22 @@ class StreamingPiCamera2(BaseCamera):
stream_resolution = lt.ThingProperty( stream_resolution = lt.ThingProperty(
tuple[int, int], tuple[int, int],
initial_value=(820, 616), initial_value=(820, 616),
description="Resolution to use for the MJPEG stream",
) )
"""Resolution to use for the MJPEG stream."""
mjpeg_bitrate = lt.ThingProperty( mjpeg_bitrate = lt.ThingProperty(
Optional[int], Optional[int],
initial_value=100000000, initial_value=100000000,
description="Bitrate for MJPEG stream (None for default)",
) )
"""Bitrate for MJPEG stream (None for default)."""
stream_active = lt.ThingProperty( stream_active = lt.ThingProperty(
bool, bool,
initial_value=False, initial_value=False,
description="Whether the MJPEG stream is active",
observable=True, observable=True,
readonly=True, readonly=True,
) )
"""Whether the MJPEG stream is active."""
def save_settings(self): def save_settings(self):
"""Override save_settings to ensure that camera properties don't recurse. """Override save_settings to ensure that camera properties don't recurse.
@ -153,7 +162,7 @@ class StreamingPiCamera2(BaseCamera):
This method is run by any Thing when a ThingSetting is saved. However, the This method is run by any Thing when a ThingSetting is saved. However, the
method reads the thing_setting. As reading the thing setting talks to the method reads the thing_setting. As reading the thing setting talks to the
camera and calls save_settings if the value is not as expected, this could camera and calls save_settings if the value is not as expected, this could
cause recursion. Aslo this means that saving one setting causes all others cause recursion. Also this means that saving one setting causes all others
to be read each time. to be read each time.
""" """
try: try:
@ -168,6 +177,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_setting @lt.thing_setting
def analogue_gain(self) -> float: def analogue_gain(self) -> float:
"""The Analogue gain applied by the camera sensor."""
if not self._setting_save_in_progress and self.streaming: if not self._setting_save_in_progress and self.streaming:
with self._streaming_picamera() as cam: with self._streaming_picamera() as cam:
cam_value = cam.capture_metadata()["AnalogueGain"] cam_value = cam.capture_metadata()["AnalogueGain"]
@ -187,6 +197,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_setting @lt.thing_setting
def colour_gains(self) -> tuple[float, float]: def colour_gains(self) -> tuple[float, float]:
"""The red and blue colour gains, must be between 0.0 and 32.0."""
if not self._setting_save_in_progress and self.streaming: if not self._setting_save_in_progress and self.streaming:
with self._streaming_picamera() as cam: with self._streaming_picamera() as cam:
cam_value = cam.capture_metadata()["ColourGains"] cam_value = cam.capture_metadata()["ColourGains"]
@ -206,6 +217,11 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_setting @lt.thing_setting
def exposure_time(self) -> int: def exposure_time(self) -> int:
"""The camera exposure time in microseconds.
When setting this property the camera will adjust the set value
to the nearest allowed value that is lower than the current setting.
"""
if not self._setting_save_in_progress and self.streaming: if not self._setting_save_in_progress and self.streaming:
with self._streaming_picamera() as cam: with self._streaming_picamera() as cam:
cam_value = cam.capture_metadata()["ExposureTime"] cam_value = cam.capture_metadata()["ExposureTime"]
@ -246,7 +262,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_property @lt.thing_property
def sensor_modes(self) -> list[SensorMode]: def sensor_modes(self) -> list[SensorMode]:
"""All the available modes the current sensor supports""" """All the available modes the current sensor supports."""
if not self._sensor_modes: if not self._sensor_modes:
with self._streaming_picamera() as cam: with self._streaming_picamera() as cam:
self._sensor_modes = cam.sensor_modes self._sensor_modes = cam.sensor_modes
@ -256,15 +272,14 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_property @lt.thing_property
def sensor_mode(self) -> Optional[SensorModeSelector]: def sensor_mode(self) -> Optional[SensorModeSelector]:
"""The intended sensor mode of the camera""" """The intended sensor mode of the camera."""
if self._sensor_mode is None: if self._sensor_mode is None:
return None return None
return SensorModeSelector(**self._sensor_mode) return SensorModeSelector(**self._sensor_mode)
@sensor_mode.setter @sensor_mode.setter
def sensor_mode(self, new_mode: Optional[SensorModeSelector | dict]): def sensor_mode(self, new_mode: Optional[SensorModeSelector | dict]):
"""Change the sensor mode used""" """Change the sensor mode used."""
if new_mode is None: if new_mode is None:
self._sensor_mode = None self._sensor_mode = None
elif isinstance(new_mode, SensorModeSelector): elif isinstance(new_mode, SensorModeSelector):
@ -280,16 +295,17 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_property @lt.thing_property
def sensor_resolution(self) -> Optional[tuple[int, int]]: def sensor_resolution(self) -> Optional[tuple[int, int]]:
"""The native resolution of the camera's sensor""" """The native resolution of the camera's sensor."""
with self._streaming_picamera() as cam: with self._streaming_picamera() as cam:
return cam.sensor_resolution return cam.sensor_resolution
tuning = lt.ThingSetting(Optional[dict], None, readonly=True) tuning = lt.ThingSetting(Optional[dict], None, readonly=True)
"""The Raspberry PiCamera Tuning File JSON."""
def _initialise_picamera(self): def _initialise_picamera(self):
"""Acquire the picamera device and store it as `self._picamera`. """Acquire the picamera device and store it as ``self._picamera``.
This duplicates logic in `Picamera2.__init__` to provide a tuning file that This duplicates logic in ``Picamera2.__init__`` to provide a tuning file that
will be read when the camera system initialises. will be read when the camera system initialises.
""" """
if self._picamera_lock is not None: if self._picamera_lock is not None:
@ -319,6 +335,11 @@ class StreamingPiCamera2(BaseCamera):
self._picamera_lock = RLock() self._picamera_lock = RLock()
def __enter__(self): def __enter__(self):
"""Start streaming when the Thing context manager is opened.
This opens the picamera connection, initialises the camera, sets the
sensor_modes property, and then starts the streams.
"""
self._initialise_picamera() self._initialise_picamera()
# populate sensor modes by reading the property # populate sensor modes by reading the property
self.sensor_modes self.sensor_modes
@ -327,21 +348,22 @@ class StreamingPiCamera2(BaseCamera):
@property @property
def streaming(self) -> bool: def streaming(self) -> bool:
"""True if the camera is streaming""" """True if the camera is streaming."""
return self._picamera is not None and self._picamera.started return self._picamera is not None and self._picamera.started
@contextmanager @contextmanager
def _streaming_picamera(self, pause_stream=False) -> Iterator[Picamera2]: def _streaming_picamera(self, pause_stream=False) -> Iterator[Picamera2]:
"""Lock access to picamera and return the underlying `Picamera2` instance. """Lock access to picamera and return the underlying ``Picamera2`` instance.
Optionally the stream can be paused to allow updating the camera settings. Optionally the stream can be paused to allow updating the camera settings.
:param pause_stream: If False the `Picamera2` instance is simply yielded. :param pause_stream: If False the ``Picamera2`` instance is simply yielded.
If True: If True:
* Stop the MJPEG Stream
* Yield the `Picamera2` instance for function calling the context manager to * Stop the MJPEG Stream
make changes. * Yield the ``Picamera2`` instance for function calling the context manager to
* On closing of the context manager the stream will restart. make changes.
* On closing of the context manager the stream will restart.
""" """
already_streaming = self.stream_active already_streaming = self.stream_active
with self._picamera_lock: with self._picamera_lock:
@ -354,7 +376,7 @@ class StreamingPiCamera2(BaseCamera):
self.start_streaming() self.start_streaming()
def __exit__(self, exc_type, exc_value, traceback): def __exit__(self, exc_type, exc_value, traceback):
# Shut down the camera """Close the picamera connection when the Thing context manager is closed."""
self.stop_streaming() self.stop_streaming()
with self._streaming_picamera() as cam: with self._streaming_picamera() as cam:
cam.close() cam.close()
@ -364,16 +386,16 @@ class StreamingPiCamera2(BaseCamera):
def start_streaming( def start_streaming(
self, main_resolution: tuple[int, int] = (820, 616), buffer_count: int = 6 self, main_resolution: tuple[int, int] = (820, 616), buffer_count: int = 6
) -> None: ) -> None:
""" """Start the MJPEG stream. This is where persistent controls are sent to camera.
Start the MJPEG stream. This is where persistent controls are sent to camera.
Sets the camera resolutions based on input parameters, and sets the low-res Sets the camera resolutions based on input parameters, and sets the low-res
resolution to (320, 240). Note: (320, 240) is a standard from the Pi Camera resolution to (320, 240). Note: (320, 240) is a standard from the Pi Camera
manual. manual.
Create two streams: Create two streams:
- `lores_mjpeg_stream` for autofocus at low-res resolution
- `mjpeg_stream` for preview. This is the `main_resolution` if this is less * ``lores_mjpeg_stream`` for autofocus at low-res resolution
* ``mjpeg_stream`` for preview. This is the ``main_resolution`` if this is less
than (1280, 960), or the low-res resolution if above. This allows for than (1280, 960), or the low-res resolution if above. This allows for
high resolution capture without streaming high resolution video. high resolution capture without streaming high resolution video.
@ -434,9 +456,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_action @lt.thing_action
def stop_streaming(self, stop_web_stream: bool = True) -> None: def stop_streaming(self, stop_web_stream: bool = True) -> None:
""" """Stop the MJPEG stream."""
Stop the MJPEG stream
"""
with self._streaming_picamera() as picam: with self._streaming_picamera() as picam:
try: try:
picam.stop_recording() # This should also stop the extra lores encoder picam.stop_recording() # This should also stop the extra lores encoder
@ -478,7 +498,7 @@ class StreamingPiCamera2(BaseCamera):
stream_name: Literal["main", "lores", "raw", "full"] = "main", stream_name: Literal["main", "lores", "raw", "full"] = "main",
wait: Optional[float] = 0.9, wait: Optional[float] = 0.9,
) -> ArrayModel: ) -> ArrayModel:
"""Acquire one image from the camera and return as an array """Acquire one image from the camera and return as an array.
This function will produce a nested list containing an uncompressed RGB image. This function will produce a nested list containing an uncompressed RGB image.
It's likely to be highly inefficient - raw and/or uncompressed captures using It's likely to be highly inefficient - raw and/or uncompressed captures using
@ -489,7 +509,6 @@ class StreamingPiCamera2(BaseCamera):
A TimeoutError is raised if this time is exceeded during capture. A TimeoutError is raised if this time is exceeded during capture.
Default = 0.9s, lower than the 1s timeout default in picamera yaml settings Default = 0.9s, lower than the 1s timeout default in picamera yaml settings
""" """
# This was slower than capture_image for our use case, but directly returning # This was slower than capture_image for our use case, but directly returning
# an image as an array is still a useful feature # an image as an array is still a useful feature
if stream_name == "full": if stream_name == "full":
@ -501,7 +520,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_property @lt.thing_property
def camera_configuration(self) -> Mapping: def camera_configuration(self) -> Mapping:
"""The "configuration" dictionary of the picamera2 object """The "configuration" dictionary of the picamera2 object.
The "configuration" sets the resolution and format of the camera's streams. The "configuration" sets the resolution and format of the camera's streams.
Together with the "tuning" it determines how the sensor is configured and Together with the "tuning" it determines how the sensor is configured and
@ -521,15 +540,15 @@ class StreamingPiCamera2(BaseCamera):
resolution: Literal["lores", "main", "full"] = "main", resolution: Literal["lores", "main", "full"] = "main",
wait: Optional[float] = 0.9, wait: Optional[float] = 0.9,
) -> JPEGBlob: ) -> JPEGBlob:
"""Acquire one image from the camera as a JPEG """Acquire one image from the camera as a JPEG.
The JPEG will be acquired using `Picamera2.capture_file`. If the The JPEG will be acquired using ``Picamera2.capture_file``. If the
`resolution` parameter is `main` or `lores`, it will be captured ``resolution`` parameter is ``main`` or ``lores``, it will be captured
from the main preview stream, or the low-res preview stream, from the main preview stream, or the low-res preview stream,
respectively. This means the camera won't be reconfigured, and respectively. This means the camera won't be reconfigured, and
the stream will not pause (though it may miss one frame). the stream will not pause (though it may miss one frame).
If `full` resolution is requested, we will briefly pause the If ``full`` resolution is requested, we will briefly pause the
MJPEG stream and reconfigure the camera to capture a full MJPEG stream and reconfigure the camera to capture a full
resolution image. resolution image.
@ -572,7 +591,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_property @lt.thing_property
def capture_metadata(self) -> dict: def capture_metadata(self) -> dict:
"""Return the metadata from the camera""" """Return the metadata from the camera."""
with self._streaming_picamera() as cam: with self._streaming_picamera() as cam:
return cam.capture_metadata() return cam.capture_metadata()
@ -582,17 +601,18 @@ class StreamingPiCamera2(BaseCamera):
target_white_level: int = 700, target_white_level: int = 700,
percentile: float = 99.9, percentile: float = 99.9,
): ):
"""Adjust exposure until a the target white level is reached """Adjust exposure until a the target white level is reached.
Starting from the minimum exposure, gradually increase exposure until Starting from the minimum exposure, gradually increase exposure until
the image reaches the specified white level. the image reaches the specified white level.
:param target_white_level: The target 10bit white level. 10-bit data has a :param target_white_level: The target 10bit white level. 10-bit data has a
theoretical maximum of 1023, but with black level correction the true maxiumum theoretical maximum of 1023, but with black level correction the true
is about 950. Default is 700 as this is approximately 70% saturated. maximum is about 950. Default is 700 as this is approximately 70%
saturated.
:param percentile: The percentile to use instead of maximum. Default 99.9. When :param percentile: The percentile to use instead of maximum. Default 99.9. When
calculating the brightest pixel, a percentile is used rather than the calculating the brightest pixel, a percentile is used rather than the
maximum in order to be robust to a small number of noisy/bright pixels. maximum in order to be robust to a small number of noisy/bright pixels.
""" """
with self._streaming_picamera(pause_stream=True) as cam: with self._streaming_picamera(pause_stream=True) as cam:
recalibrate_utils.adjust_shutter_and_gain_from_raw( recalibrate_utils.adjust_shutter_and_gain_from_raw(
@ -607,7 +627,7 @@ class StreamingPiCamera2(BaseCamera):
method: Literal["percentile", "centre"] = "centre", method: Literal["percentile", "centre"] = "centre",
luminance_power: float = 1.0, luminance_power: float = 1.0,
): ):
"""Correct the white balance of the image """Correct the white balance of the image.
This calibration requires a neutral image, such that the 99th centile This calibration requires a neutral image, such that the 99th centile
of each colour channel should correspond to white. We calculate the of each colour channel should correspond to white. We calculate the
@ -615,7 +635,7 @@ class StreamingPiCamera2(BaseCamera):
image with the lens shading correction applied, which should mean image with the lens shading correction applied, which should mean
that the image is uniform, rather than weighted towards the centre. that the image is uniform, rather than weighted towards the centre.
If `method` is `"centre"`, we will correct the mean of the central 10% If ``method`` is ``"centre"``, we will correct the mean of the central 10%
of the image. of the image.
""" """
with self._streaming_picamera(pause_stream=True) as cam: with self._streaming_picamera(pause_stream=True) as cam:
@ -657,7 +677,7 @@ class StreamingPiCamera2(BaseCamera):
def colour_correction_matrix( def colour_correction_matrix(
self, self,
) -> tuple[float, float, float, float, float, float, float, float, float]: ) -> tuple[float, float, float, float, float, float, float, float, float]:
"""The `colour_correction_matrix` from the tuning file. """The ``colour_correction_matrix`` from the tuning file.
This is broken out into its own property for convenience and compatibility with This is broken out into its own property for convenience and compatibility with
the micromanager API the micromanager API
@ -679,8 +699,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_action @lt.thing_action
def reset_ccm(self): def reset_ccm(self):
""" """Overwrite the colour correction matrix in camera tuning with default values.
Overwrite the colour correction matrix in camera tuning with default values.
These values are from the Raspberry Pi Camera Algorithm and Tuning Guide, page These values are from the Raspberry Pi Camera Algorithm and Tuning Guide, page
45. 45.
@ -716,15 +735,15 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_action @lt.thing_action
def full_auto_calibrate(self) -> None: def full_auto_calibrate(self) -> None:
"""Perform a full auto-calibration """Perform a full auto-calibration.
This function will call the other calibration actions in sequence: This function will call the other calibration actions in sequence:
* `flat_lens_shading` to disable flat-field * ``flat_lens_shading`` to disable flat-field
* `auto_expose_from_minimum` * ``auto_expose_from_minimum``
* `set_static_green_equalisation` to set geq offset to max * ``set_static_green_equalisation`` to set geq offset to max
* `calibrate_lens_shading` * ``calibrate_lens_shading``
* `calibrate_white_balance` * ``calibrate_white_balance``
""" """
self.flat_lens_shading() self.flat_lens_shading()
self.auto_expose_from_minimum() self.auto_expose_from_minimum()
@ -734,13 +753,13 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_action @lt.thing_action
def flat_lens_shading(self) -> None: def flat_lens_shading(self) -> None:
"""Disable flat-field correction """Disable flat-field correction.
This method will set a completely flat lens shading table. It is not the This method will set a completely flat lens shading table. It is not the
same as the default behaviour, which is to use an adaptive lens shading same as the default behaviour, which is to use an adaptive lens shading
table. table.
This flat table is used to take an image wth no lens shading so that the This flat table is used to take an image with no lens shading so that the
correct lens shading table can be calibrated. correct lens shading table can be calibrated.
""" """
with self._streaming_picamera(pause_stream=True): with self._streaming_picamera(pause_stream=True):
@ -753,7 +772,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_property @lt.thing_property
def lens_shading_tables(self) -> Optional[LensShading]: def lens_shading_tables(self) -> Optional[LensShading]:
"""The current lens shading (i.e. flat-field correction) """The current lens shading (i.e. flat-field correction).
Return the current lens shading correction, as three 2D lists each with Return the current lens shading correction, as three 2D lists each with
dimensions 16x12, if a static lens shading table is in use. dimensions 16x12, if a static lens shading table is in use.
@ -775,7 +794,7 @@ class StreamingPiCamera2(BaseCamera):
return None return None
def reshape_lst(lin: list[float]) -> list[list[float]]: def reshape_lst(lin: list[float]) -> list[list[float]]:
"""Reshape the 192 element list into a 2D 16x12 list""" """Reshape the 192 element list into a 2D 16x12 list."""
w, h = 16, 12 w, h = 16, 12
return [lin[w * i : w * (i + 1)] for i in range(h)] return [lin[w * i : w * (i + 1)] for i in range(h)]
@ -787,7 +806,7 @@ class StreamingPiCamera2(BaseCamera):
@lens_shading_tables.setter @lens_shading_tables.setter
def lens_shading_tables(self, lst: LensShading) -> None: def lens_shading_tables(self, lst: LensShading) -> None:
"""Set the lens shading tables""" """Set the lens shading tables."""
with self._streaming_picamera(pause_stream=True): with self._streaming_picamera(pause_stream=True):
recalibrate_utils.set_static_lst( recalibrate_utils.set_static_lst(
self.tuning, self.tuning,
@ -800,19 +819,19 @@ class StreamingPiCamera2(BaseCamera):
def correct_colour_gains_for_lens_shading( def correct_colour_gains_for_lens_shading(
self, colour_gains: tuple[float, float] self, colour_gains: tuple[float, float]
) -> tuple[float, float]: ) -> tuple[float, float]:
"""Correct white balance gains for the effect of lens shading """Correct white balance gains for the effect of lens shading.
The white balance algorithm we use assumes the brightest pixels The white balance algorithm we use assumes the brightest pixels
should be white, and that the only thing affecting the colour of should be white, and that the only thing affecting the colour of
said pixels is the `colour_gains`. said pixels is the ``colour_gains``.
The lens shading correction is normalised such that the *minimum* The lens shading correction is normalised such that the *minimum*
gain in the `Cr` and `Cb` channels is 1. The white balance gain in the ``Cr`` and ``Cb`` channels is 1. The white balance
assumption above requires that the gain for the brightest pixels assumption above requires that the gain for the brightest pixels
is 1. The solution might be that, when calibrating, we note which is 1. The solution might be that, when calibrating, we note which
pixels are brightest (usually the centre) and explicitly use pixels are brightest (usually the centre) and explicitly use
the LST values for there. However, for now I will assume that we the LST values for there. However, for now I will assume that we
need to normalise by the **maximum** of the `Cr` and `Cb` need to normalise by the **maximum** of the ``Cr`` and ``Cb``
channels, which is correct the majority of the time. channels, which is correct the majority of the time.
""" """
if not self.lens_shading_is_static: if not self.lens_shading_is_static:
@ -830,7 +849,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_action @lt.thing_action
def flat_lens_shading_chrominance(self) -> None: def flat_lens_shading_chrominance(self) -> None:
"""Disable flat-field correction """Disable flat-field correction.
This method will set the chrominance of the lens shading table to be This method will set the chrominance of the lens shading table to be
flat, i.e. we'll correct vignetting of intensity, but not any change in flat, i.e. we'll correct vignetting of intensity, but not any change in
@ -845,7 +864,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_action @lt.thing_action
def reset_lens_shading(self) -> None: def reset_lens_shading(self) -> None:
"""Revert to default lens shading settings """Revert to default lens shading settings.
This method will restore the default "adaptive" lens shading method used This method will restore the default "adaptive" lens shading method used
by the Raspberry Pi camera. by the Raspberry Pi camera.
@ -856,7 +875,7 @@ class StreamingPiCamera2(BaseCamera):
@lt.thing_property @lt.thing_property
def lens_shading_is_static(self) -> bool: def lens_shading_is_static(self) -> bool:
"""Whether the lens shading is static """Whether the lens shading is static.
This property is true if the lens shading correction has been set to use This property is true if the lens shading correction has been set to use
a static table (i.e. the number of automatic correction iterations is zero). a static table (i.e. the number of automatic correction iterations is zero).

118
src/openflexure_microscope_server/things/camera/picamera_recalibrate_utils.py
View file

@ -1,9 +1,8 @@
""" """Functions to set up a Raspberry Pi Camera v2 for scientific use.
Functions to set up a Raspberry Pi Camera v2 for scientific use
This module provides slower, simpler functions to set the This module provides slower, simpler functions to set the
gain, exposure, and white balance of a Raspberry Pi camera, using gain, exposure, and white balance of a Raspberry Pi camera, using
the `picamera2` Python library. It's mostly used by the OpenFlexure the ``picamera2`` Python library. It's mostly used by the OpenFlexure
Microscope, though it deliberately has no hard dependencies on Microscope, though it deliberately has no hard dependencies on
said software, so that it's useful on its own. said software, so that it's useful on its own.
@ -20,14 +19,15 @@ to "memory" or nonlinearities in the camera's image processing
pipeline, is to use raw images. This is quite slow, but very pipeline, is to use raw images. This is quite slow, but very
reliable. The three steps above can be accomplished by: reliable. The three steps above can be accomplished by:
``` .. code-block:: python
picamera = picamera2.Picamera2()
picamera = picamera2.Picamera2()
adjust_shutter_and_gain_from_raw(picamera)
adjust_white_balance_from_raw(picamera)
lst = lst_from_camera(picamera)
picamera.lens_shading_table = lst
adjust_shutter_and_gain_from_raw(picamera)
adjust_white_balance_from_raw(picamera)
lst = lst_from_camera(picamera)
picamera.lens_shading_table = lst
```
""" """
# Disable N806 & 803, which checks that all variables and args are lowercase. # Disable N806 & 803, which checks that all variables and args are lowercase.
@ -52,10 +52,10 @@ LensShadingTables = tuple[np.ndarray, np.ndarray, np.ndarray]
def load_default_tuning(cam: Picamera2) -> dict: def load_default_tuning(cam: Picamera2) -> dict:
"""Load the default tuning file for the camera """Load the default tuning file for the camera.
This will open and close the camera to determine its model. If you are This will open and close the camera to determine its model. If you are
using a model that's supported by `picamera2` it should have a tuning using a model that's supported by ``picamera2`` it should have a tuning
file built in. If not, this will probably crash with an error. file built in. If not, this will probably crash with an error.
Error handling for unsupported cameras is not something we are likely Error handling for unsupported cameras is not something we are likely
@ -76,7 +76,7 @@ def load_default_tuning(cam: Picamera2) -> dict:
def set_minimum_exposure(camera: Picamera2) -> None: def set_minimum_exposure(camera: Picamera2) -> None:
"""Enable manual exposure, with low gain and shutter speed """Enable manual exposure, with low gain and shutter speed.
We set exposure mode to manual, analog and digital gain We set exposure mode to manual, analog and digital gain
to 1, and shutter speed to the minimum (8us for Pi Camera v2) to 1, and shutter speed to the minimum (8us for Pi Camera v2)
@ -84,7 +84,7 @@ def set_minimum_exposure(camera: Picamera2) -> None:
Note ISO is left at auto, because this is needed for the gains Note ISO is left at auto, because this is needed for the gains
to be set correctly. to be set correctly.
""" """
# Disable Automatic exposure and gain algoritm (AeEnable), and set analogue # Disable Automatic exposure and gain algorithm (AeEnable), and set analogue
# gain and exposure time. # gain and exposure time.
# Setting the shutter speed to 1us will result in it being set # Setting the shutter speed to 1us will result in it being set
# to the minimum possible, which is ~8us for PiCamera v2 # to the minimum possible, which is ~8us for PiCamera v2
@ -93,7 +93,7 @@ def set_minimum_exposure(camera: Picamera2) -> None:
class ExposureTest(BaseModel): class ExposureTest(BaseModel):
"""Record the results of testing the camera's current exposure settings""" """Record the results of testing the camera's current exposure settings."""
level: int level: int
exposure_time: int exposure_time: int
@ -101,7 +101,7 @@ class ExposureTest(BaseModel):
def test_exposure_settings(camera: Picamera2, percentile: float) -> ExposureTest: def test_exposure_settings(camera: Picamera2, percentile: float) -> ExposureTest:
"""Evaluate current exposure settings using a raw image """Evaluate current exposure settings using a raw image.
CAMERA SHOULD BE STARTED! CAMERA SHOULD BE STARTED!
@ -136,7 +136,7 @@ def test_exposure_settings(camera: Picamera2, percentile: float) -> ExposureTest
def check_convergence(test: ExposureTest, target: int, tolerance: float) -> bool: def check_convergence(test: ExposureTest, target: int, tolerance: float) -> bool:
"""Check whether the brightness is within the specified target range""" """Check whether the brightness is within the specified target range."""
return abs(test.level - target) < target * tolerance return abs(test.level - target) < target * tolerance
@ -152,25 +152,19 @@ def adjust_shutter_and_gain_from_raw(
This routine is slow but effective. It uses raw images, so we This routine is slow but effective. It uses raw images, so we
are not affected by white balance or digital gain. are not affected by white balance or digital gain.
:param camera: A Picamera2 object.
:param target_white_level: The raw, 10-bit value we aim for. The brightest pixels
should be approximately this bright. Maximum possible is about 900, 700 is
reasonable.
:param max_iterations: We will terminate once we perform this many iterations,
whether or not we converge. More than 10 shouldn't happen.
:param tolerance: How close to the target value we consider "done". Expressed as a
fraction of the ``target_white_level`` so 0.05 means +/- 5%
:param percentile: Rather then use the maximum value for each channel, we calculate
a percentile. This makes us robust to single pixels that are bright/noisy.
99.9% still picks the top of the brightness range, but seems much more reliable
than just ``np.max()``.
Arguments:
target_white_level:
The raw, 10-bit value we aim for. The brightest pixels
should be approximately this bright. Maximum possible
is about 900, 700 is reasonable.
max_iterations:
We will terminate once we perform this many iterations,
whether or not we converge. More than 10 shouldn't happen.
tolerance:
How close to the target value we consider "done". Expressed
as a fraction of the ``target_white_level`` so 0.05 means
+/- 5%
percentile:
Rather then use the maximum value for each channel, we
calculate a percentile. This makes us robust to single
pixels that are bright/noisy. 99.9% still picks the top
of the brightness range, but seems much more reliable
than just ``np.max()``.
""" """
# TODO: read black level and bit depth from camera? # TODO: read black level and bit depth from camera?
if target_white_level * (tolerance + 1) >= 959: if target_white_level * (tolerance + 1) >= 959:
@ -327,7 +321,7 @@ def channels_from_bayer_array(bayer_array: np.ndarray) -> np.ndarray:
def get_16x12_grid(chan: np.ndarray, dx: int, dy: int) -> np.ndarray: def get_16x12_grid(chan: np.ndarray, dx: int, dy: int) -> np.ndarray:
"""Compresses channel down to a 16x12 grid - from libcamera """Compresses channel down to a 16x12 grid - from libcamera.
This is taken from This is taken from
https://git.linuxtv.org/libcamera.git/tree/utils/raspberrypi/ctt/ctt_alsc.py https://git.linuxtv.org/libcamera.git/tree/utils/raspberrypi/ctt/ctt_alsc.py
@ -349,9 +343,9 @@ def get_16x12_grid(chan: np.ndarray, dx: int, dy: int) -> np.ndarray:
def upsample_channels(grids: np.ndarray, shape: tuple[int]) -> np.ndarray: def upsample_channels(grids: np.ndarray, shape: tuple[int]) -> np.ndarray:
"""Zoom an image in the last two dimensions """Zoom an image in the last two dimensions.
This is effectively the inverse operation of `get_16x12_grid` This is effectively the inverse operation of ``get_16x12_grid``
""" """
zoom_factors = [ zoom_factors = [
1, 1,
@ -360,7 +354,7 @@ def upsample_channels(grids: np.ndarray, shape: tuple[int]) -> np.ndarray:
def downsampled_channels(channels: np.ndarray, blacklevel=64) -> list[np.ndarray]: def downsampled_channels(channels: np.ndarray, blacklevel=64) -> list[np.ndarray]:
"""Generate a downsampled, un-normalised image from which to calculate the LST """Generate a downsampled, un-normalised image from which to calculate the LST.
TODO: blacklevel probably ought to be determined from the camera... TODO: blacklevel probably ought to be determined from the camera...
""" """
@ -381,22 +375,21 @@ def downsampled_channels(channels: np.ndarray, blacklevel=64) -> list[np.ndarray
def lst_from_channels(channels: np.ndarray) -> LensShadingTables: def lst_from_channels(channels: np.ndarray) -> LensShadingTables:
"""Given the 4 Bayer colour channels from a white image, generate a LST. """Given the 4 Bayer colour channels from a white image, generate a LST.
Internally, is just calls `downsampled_channels` and `lst_from_grids`. Internally, is just calls ``downsampled_channels`` and ``lst_from_grids``.
""" """
grids = downsampled_channels(channels) grids = downsampled_channels(channels)
return lst_from_grids(grids) return lst_from_grids(grids)
def lst_from_grids(grids: np.ndarray) -> LensShadingTables: def lst_from_grids(grids: np.ndarray) -> LensShadingTables:
"""Given 4 downsampled grids, generate the luminance and chrominance tables """Given 4 downsampled grids, generate the luminance and chrominance tables.
The grids are the 4 BAYER channels RGGB The grids are the 4 BAYER channels RGGB
The LST format has changed with `picamera2` and now uses a fixed resolution, The LST format has changed with ``picamera2`` and now uses a fixed resolution,
and is in luminance, Cr, Cb format. This function returns three ndarrays of and is in luminance, Cr, Cb format. This function returns three ndarrays of
luminance, Cr, Cb, each with shape (12, 16). luminance, Cr, Cb, each with shape (12, 16).
""" """
# Calculated red, green, and blue channels from Bayer data # Calculated red, green, and blue channels from Bayer data
r: np.ndarray = grids[3, ...] r: np.ndarray = grids[3, ...]
g: np.ndarray = np.mean(grids[1:3, ...], axis=0) g: np.ndarray = np.mean(grids[1:3, ...], axis=0)
@ -415,11 +408,11 @@ def lst_from_grids(grids: np.ndarray) -> LensShadingTables:
def grids_from_lst(lum: np.ndarray, Cr: np.ndarray, Cb: np.ndarray) -> np.ndarray: def grids_from_lst(lum: np.ndarray, Cr: np.ndarray, Cb: np.ndarray) -> np.ndarray:
"""Convert form luminance/chrominance dict to four RGGB channels """Convert form luminance/chrominance dict to four RGGB channels.
Note that these will be normalised - the maximum green value is always 1. Note that these will be normalised - the maximum green value is always 1.
Also, note that the channels are BGGR, to be consistent with the Also, note that the channels are BGGR, to be consistent with the
`channels_from_raw_image` function. This should probably change in the ``channels_from_raw_image`` function. This should probably change in the
future. future.
""" """
G = 1 / np.array(lum) G = 1 / np.array(lum)
@ -434,9 +427,9 @@ def set_static_lst(
cr: np.ndarray, cr: np.ndarray,
cb: np.ndarray, cb: np.ndarray,
) -> None: ) -> None:
"""Update the `rpi.alsc` section of a camera tuning dict to use a static correcton. """Update the ``rpi.alsc`` section of a camera tuning dict to use a static correction.
`tuning` will be updated in-place to set its shading to static, and disable any ``tuning`` will be updated in-place to set its shading to static, and disable any
adaptive tweaking by the algorithm. adaptive tweaking by the algorithm.
""" """
for table in luminance, cr, cb: for table in luminance, cr, cb:
@ -459,9 +452,9 @@ def set_static_ccm(
float, float, float, float, float, float, float, float, float float, float, float, float, float, float, float, float, float
], ],
) -> None: ) -> None:
"""Update the `rpi.alsc` section of a camera tuning dict to use a static correcton. """Update the ``rpi.alsc`` section of a camera tuning dict to use a static correction.
`tuning` will be updated in-place to set its shading to static, and disable any ``tuning`` will be updated in-place to set its shading to static, and disable any
adaptive tweaking by the algorithm. adaptive tweaking by the algorithm.
""" """
ccm = Picamera2.find_tuning_algo(tuning, "rpi.ccm") ccm = Picamera2.find_tuning_algo(tuning, "rpi.ccm")
@ -469,13 +462,13 @@ def set_static_ccm(
def get_static_ccm(tuning: dict) -> None: def get_static_ccm(tuning: dict) -> None:
"""Get the `rpi.ccm` section of a camera tuning dict""" """Get the ``rpi.ccm`` section of a camera tuning dict."""
ccm = Picamera2.find_tuning_algo(tuning, "rpi.ccm") ccm = Picamera2.find_tuning_algo(tuning, "rpi.ccm")
return ccm["ccms"] return ccm["ccms"]
def lst_is_static(tuning: dict) -> bool: def lst_is_static(tuning: dict) -> bool:
"""Whether the lens shading table is set to static""" """Whether the lens shading table is set to static."""
alsc = Picamera2.find_tuning_algo(tuning, "rpi.alsc") alsc = Picamera2.find_tuning_algo(tuning, "rpi.alsc")
return alsc["n_iter"] == 0 return alsc["n_iter"] == 0
@ -484,26 +477,29 @@ def set_static_geq(
tuning: dict, tuning: dict,
offset: int = 65535, offset: int = 65535,
) -> None: ) -> None:
"""Update the `rpi.geq` section of a camera tuning dict to always use green """Update the ``rpi.geq`` section of a camera tuning dict.
equalisation that averages the green pixels in the red and blue rows.
`tuning` will be updated in-place to set the geq offest to the given value. :param tuning: the raspberry pi tuning file. This will be updated in-place to
The default 65535 is the maximum allowed value. This means set the geq offset to the given value.
the brightness will always be below the threshold where averaging is used. :param offset: The desired green equalisation offset. Default 65535. The default is
the maximum allowed value. This means the brightness will always be below the
threshold where averaging is used. This is default as we always need the green
equalisation to averages the green pixels in the red and blue rows due to the
chief ray angle compensation issue when the the stock lens is replaced by an
objective.
""" """
geq = Picamera2.find_tuning_algo(tuning, "rpi.geq") geq = Picamera2.find_tuning_algo(tuning, "rpi.geq")
geq["offset"] = offset # max out offset to disable the adaptive green equalisation geq["offset"] = offset # max out offset to disable the adaptive green equalisation
def _geq_is_static(tuning: dict) -> bool: def _geq_is_static(tuning: dict) -> bool:
"""Whether the green equalisation is set to static""" """Whether the green equalisation is set to static."""
geq = Picamera2.find_tuning_algo(tuning, "rpi.geq") geq = Picamera2.find_tuning_algo(tuning, "rpi.geq")
return geq["offset"] == 65535 return geq["offset"] == 65535
def index_of_algorithm(algorithms: list[dict], algorithm: str) -> int: def index_of_algorithm(algorithms: list[dict], algorithm: str) -> int:
"""Find the index of an algorithm's section in the tuning file""" """Find the index of an algorithm's section in the tuning file."""
for i, a in enumerate(algorithms): for i, a in enumerate(algorithms):
if algorithm in a: if algorithm in a:
return i return i
@ -511,7 +507,7 @@ def index_of_algorithm(algorithms: list[dict], algorithm: str) -> int:
def copy_alsc_section(from_tuning: dict, to_tuning: dict) -> None: def copy_alsc_section(from_tuning: dict, to_tuning: dict) -> None:
"""Copy the `rpi.alsc` algorithm from one tuning to another. """Copy the ``rpi.alsc`` algorithm from one tuning to another.
This is done in-place, i.e. modifying to_tuning. This is done in-place, i.e. modifying to_tuning.
""" """
@ -561,5 +557,5 @@ def recreate_camera_manager() -> None:
def as_flat_rounded_list(array: np.ndarray, round_to: int = 3) -> list[float]: def as_flat_rounded_list(array: np.ndarray, round_to: int = 3) -> list[float]:
"""Flatten array, round, and then convert to list""" """Flatten array, round, and then convert to list."""
return np.reshape(array, -1).round(round_to).tolist() return np.reshape(array, -1).round(round_to).tolist()

46
src/openflexure_microscope_server/things/camera/simulation.py
View file

@ -1,4 +1,4 @@
"""OpenFlexure Microscope OpenCV Camera """OpenFlexure Microscope OpenCV Camera.
This module defines a Thing that is responsible for using the stage and This module defines a Thing that is responsible for using the stage and
camera together to perform an autofocus routine. camera together to perform an autofocus routine.
@ -30,7 +30,7 @@ RATIO = 0.2
class SimulatedCamera(BaseCamera): class SimulatedCamera(BaseCamera):
"""A Thing representing an OpenCV camera""" """A Thing that simulates a camera for testing."""
_stage: Optional[BaseStage] = None _stage: Optional[BaseStage] = None
_server: Optional[lt.ThingServer] = None _server: Optional[lt.ThingServer] = None
@ -42,6 +42,16 @@ class SimulatedCamera(BaseCamera):
canvas_shape: tuple[int, int, int] = (3000, 4000, 3), canvas_shape: tuple[int, int, int] = (3000, 4000, 3),
frame_interval: float = 0.1, frame_interval: float = 0.1,
): ):
"""Initialise the simulated with settings for how images are generated.
:param shape: The shape (size) of the generated image.
:param glyph_shape: The size randomly positioned glyphs.
:param canvas_shape: The shape (size) of the canvas generated on initialisation
that images are cropped from. If this is too large the it uses resources,
but its size limits the range of motion of the simulation.
:param frame_interval: Nominally the time between frames on the MJPEG stream,
however the rate may be slower due to calculation time for focus.
"""
self.shape = shape self.shape = shape
self.glyph_shape = glyph_shape self.glyph_shape = glyph_shape
self.canvas_shape = canvas_shape self.canvas_shape = canvas_shape
@ -53,7 +63,7 @@ class SimulatedCamera(BaseCamera):
self.generate_canvas() self.generate_canvas()
def generate_sprites(self): def generate_sprites(self):
"""Generate sprites to populate the image""" """Generate sprites to populate the image."""
self.sprites = [] self.sprites = []
black = np.zeros(self.glyph_shape, dtype=np.uint8) black = np.zeros(self.glyph_shape, dtype=np.uint8)
x = np.arange(black.shape[0]) x = np.arange(black.shape[0])
@ -65,7 +75,7 @@ class SimulatedCamera(BaseCamera):
self.sprites.append(sprite) self.sprites.append(sprite)
def generate_blobs(self, n_blobs: int = 1000): def generate_blobs(self, n_blobs: int = 1000):
"""Generate coordinates of blobs """Generate coordinates of blobs.
Blobs are characterised by X, Y, sprite Blobs are characterised by X, Y, sprite
We also generate a KD tree to rapidly find blobs in an image We also generate a KD tree to rapidly find blobs in an image
@ -78,7 +88,7 @@ class SimulatedCamera(BaseCamera):
self.blobs[:, 2] = rng.choice(len(self.sprites), n_blobs) self.blobs[:, 2] = rng.choice(len(self.sprites), n_blobs)
def generate_canvas(self): def generate_canvas(self):
"""Generate a blank canvas""" """Generate a blank canvas."""
self.canvas = np.zeros(self.canvas_shape, dtype=np.uint8) self.canvas = np.zeros(self.canvas_shape, dtype=np.uint8)
self.canvas[...] = 255 self.canvas[...] = 255
w, h, _ = self.glyph_shape w, h, _ = self.glyph_shape
@ -89,7 +99,7 @@ class SimulatedCamera(BaseCamera):
] -= self.sprites[int(sprite)] ] -= self.sprites[int(sprite)]
def generate_image(self, pos: tuple[int, int, int]): def generate_image(self, pos: tuple[int, int, int]):
"""Generate an image with blobs based on supplied coordinates""" """Generate an image with blobs based on supplied coordinates."""
canvas_width, canvas_height, _ = self.canvas_shape canvas_width, canvas_height, _ = self.canvas_shape
image_width, image_height, _ = self.shape image_width, image_height, _ = self.shape
pos = tuple(x * RATIO for x in pos) pos = tuple(x * RATIO for x in pos)
@ -115,16 +125,26 @@ class SimulatedCamera(BaseCamera):
def attach_to_server( def attach_to_server(
self, server: lt.ThingServer, path: str, setting_storage_path: str self, server: lt.ThingServer, path: str, setting_storage_path: str
): ):
"""Wrap the attach_to_server method so the server instance can be stored.
Direct access to the server instance is needed to get the stage position while
maintaining the same public API as a real camera that doesn't need this access.
"""
self._server = server self._server = server
return super().attach_to_server(server, path, setting_storage_path) return super().attach_to_server(server, path, setting_storage_path)
def get_stage_position(self): def get_stage_position(self):
"""Return the stage position.
The simulation camera has access to the stage position so it can generate a
different image as the stage moves.
"""
if not self._stage and self._server: if not self._stage and self._server:
self._stage = self._server.things["/stage/"] self._stage = self._server.things["/stage/"]
return self._stage.instantaneous_position return self._stage.instantaneous_position
def generate_frame(self): def generate_frame(self):
"""Generate a frame with blobs based on the stage coordinates""" """Generate a frame with blobs based on the stage coordinates."""
try: try:
pos = self.get_stage_position() pos = self.get_stage_position()
except Exception as e: except Exception as e:
@ -133,19 +153,21 @@ class SimulatedCamera(BaseCamera):
return self.generate_image((pos["y"], pos["x"], pos["z"])) return self.generate_image((pos["y"], pos["x"], pos["z"]))
def __enter__(self): def __enter__(self):
"""Start the capture thread when the Thing context manager is opened."""
self._capture_enabled = True self._capture_enabled = True
self._capture_thread = Thread(target=self._capture_frames) self._capture_thread = Thread(target=self._capture_frames)
self._capture_thread.start() self._capture_thread.start()
return self return self
def __exit__(self, _exc_type, _exc_value, _traceback): def __exit__(self, _exc_type, _exc_value, _traceback):
"""Close the capture thread when the Thing context manager is closed."""
if self.stream_active: if self.stream_active:
self._capture_enabled = False self._capture_enabled = False
self._capture_thread.join() self._capture_thread.join()
@lt.thing_property @lt.thing_property
def stream_active(self) -> bool: def stream_active(self) -> bool:
"Whether the MJPEG stream is active" """Whether the MJPEG stream is active."""
if self._capture_enabled and self._capture_thread: if self._capture_enabled and self._capture_thread:
return self._capture_thread.is_alive() return self._capture_thread.is_alive()
return False return False
@ -170,13 +192,13 @@ class SimulatedCamera(BaseCamera):
self, self,
resolution: Literal["main", "full"] = "full", resolution: Literal["main", "full"] = "full",
) -> ArrayModel: ) -> ArrayModel:
"""Acquire one image from the camera and return as an array """Acquire one image from the camera and return as an array.
This function will produce a nested list containing an uncompressed RGB image. This function will produce a nested list containing an uncompressed RGB image.
It's likely to be highly inefficient - raw and/or uncompressed captures using It's likely to be highly inefficient - raw and/or uncompressed captures using
binary image formats will be added in due course. binary image formats will be added in due course.
""" """
logging.warning(f"Simulation camera doen't respect {resolution} setting") logging.warning(f"Simulation camera doesn't respect {resolution} setting")
return self.generate_frame() return self.generate_frame()
@lt.thing_action @lt.thing_action
@ -185,11 +207,11 @@ class SimulatedCamera(BaseCamera):
metadata_getter: lt.deps.GetThingStates, metadata_getter: lt.deps.GetThingStates,
resolution: Literal["main", "full"] = "main", resolution: Literal["main", "full"] = "main",
) -> JPEGBlob: ) -> JPEGBlob:
"""Acquire one image from the camera and return as a JPEG blob """Acquire one image from the camera and return as a JPEG blob.
This function will produce a JPEG image. This function will produce a JPEG image.
""" """
logging.warning(f"Simulation camera doen't respect {resolution} setting") logging.warning(f"Simulation camera doesn't respect {resolution} setting")
frame = self.capture_array() frame = self.capture_array()
jpeg = cv2.imencode(".jpg", frame)[1].tobytes() jpeg = cv2.imencode(".jpg", frame)[1].tobytes()
exif_dict = { exif_dict = {

85
src/openflexure_microscope_server/things/camera_stage_mapping.py
View file

@ -1,5 +1,4 @@
""" """OpenFlexure Microscope API extension for stage calibration.
OpenFlexure Microscope API extension for stage calibration
This file contains the HTTP API for camera/stage calibration. It This file contains the HTTP API for camera/stage calibration. It
includes calibration functions that measure the relationship between includes calibration functions that measure the relationship between
@ -46,6 +45,11 @@ XYCoordinateType = Tuple[float, float]
class HardwareInterfaceModel(BaseModel): class HardwareInterfaceModel(BaseModel):
"""A pydantic base model for a stage and camera interface.
This code has been flagged as confusing and will be replaced soon, see #476.
"""
move: Callable[[NDArray], None] move: Callable[[NDArray], None]
get_position: Callable[[], NDArray] get_position: Callable[[], NDArray]
grab_image: Callable[[], NDArray] grab_image: Callable[[], NDArray]
@ -54,10 +58,10 @@ class HardwareInterfaceModel(BaseModel):
def downsample(factor: int, image: np.ndarray) -> np.ndarray: def downsample(factor: int, image: np.ndarray) -> np.ndarray:
"""Downsample an image by taking the mean of each nxn region """Downsample an image by taking the mean of each nxn region.
This should be very efficient: we calculate the mean of each This should be very efficient: we calculate the mean of each
`factor * factor` square, no interpolation. If the image is ``factor * factor`` square, no interpolation. If the image is
not an integer multiple of the resampling factor, we discard not an integer multiple of the resampling factor, we discard
the left-over pixels. This avoids odd edge effects and keeps the left-over pixels. This avoids odd edge effects and keeps
performance quick. performance quick.
@ -80,7 +84,7 @@ DEFAULT_SETTLING_TIME = 0.2
def make_hardware_interface( def make_hardware_interface(
stage: Stage, camera: Camera, downsample_factor: int = 2 stage: Stage, camera: Camera, downsample_factor: int = 2
) -> HardwareInterfaceModel: ) -> HardwareInterfaceModel:
"""Construct the functions we need to interface with the hardware""" """Construct the functions we need to interface with the hardware."""
axes = stage.axis_names axes = stage.axis_names
def pos2dict(pos: Sequence[float]) -> Mapping[str, float]: def pos2dict(pos: Sequence[float]) -> Mapping[str, float]:
@ -124,6 +128,14 @@ HardwareInterfaceDep = Annotated[
class MoveHistory(NamedTuple): class MoveHistory(NamedTuple):
"""A named tuple containing the position over time for a single move.
This a named tuple with elements:
* ``times``
* ``stage_positions``
"""
times: List[float] times: List[float]
stage_positions: List[CoordinateType] stage_positions: List[CoordinateType]
@ -140,12 +152,16 @@ class LoggingMoveWrapper:
""" """
def __init__(self, move_function: Callable): def __init__(self, move_function: Callable):
"""Set the movement function to be wrapped.
:param move_function: the movement function to be wrapped
"""
self._move_function: Callable = move_function self._move_function: Callable = move_function
self._current_position: Optional[CoordinateType] = None self._current_position: Optional[CoordinateType] = None
self.clear_history() self.clear_history()
def __call__(self, new_position: CoordinateType, *args, **kwargs): def __call__(self, new_position: CoordinateType, *args, **kwargs):
"""Move to a new position, and record it""" """Move to a new position, and record it."""
self._history.append((time.time(), self._current_position)) self._history.append((time.time(), self._current_position))
self._move_function(new_position, *args, **kwargs) self._move_function(new_position, *args, **kwargs)
self._current_position = new_position self._current_position = new_position
@ -153,18 +169,26 @@ class LoggingMoveWrapper:
@property @property
def history(self) -> MoveHistory: def history(self) -> MoveHistory:
"""The history, as a numpy array of times and another of positions""" """The history, as a numpy array of times and another of positions."""
times: List[float] = [t for t, p in self._history if p is not None] times: List[float] = [t for t, p in self._history if p is not None]
positions: List[CoordinateType] = [p for t, p in self._history if p is not None] positions: List[CoordinateType] = [p for t, p in self._history if p is not None]
return MoveHistory(times, positions) return MoveHistory(times, positions)
def clear_history(self): def clear_history(self):
"""Reset our history to be an empty list""" """Reset our history to be an empty list."""
self._history: List[Tuple[float, Optional[CoordinateType]]] = [] self._history: List[Tuple[float, Optional[CoordinateType]]] = []
class CSMUncalibratedError(HTTPException): class CSMUncalibratedError(HTTPException):
"""An HTTP Exception raised if camera stage mapping data is needed but unavailable.
Camera Stage Mapping data is needed to convert from distances specified in fractions
of the field of view to distances in motor steps. This is used when clicking on the
live preview to move, or when performing a scan.
"""
def __init__(self): def __init__(self):
"""Customise the default error code and message of HTTPException."""
HTTPException.__init__( HTTPException.__init__(
self, self,
503, 503,
@ -176,7 +200,7 @@ class CSMUncalibratedError(HTTPException):
class CameraStageMapper(lt.Thing): class CameraStageMapper(lt.Thing):
"""A Thing to manage mapping between image and stage coordinates""" """A Thing to manage mapping between image and stage coordinates."""
@lt.thing_action @lt.thing_action
def calibrate_1d( def calibrate_1d(
@ -186,10 +210,9 @@ class CameraStageMapper(lt.Thing):
logger: lt.deps.InvocationLogger, logger: lt.deps.InvocationLogger,
direction: Tuple[float, float, float], direction: Tuple[float, float, float],
) -> DenumpifyingDict: ) -> DenumpifyingDict:
"""Move a microscope's stage in 1D, and figure out the relationship with the camera""" """Move a microscope's stage in 1D, and figure out the relationship with the camera."""
move = LoggingMoveWrapper( # log positions and times for stage calibration
hw.move move = LoggingMoveWrapper(hw.move)
) # log positions and times for stage calibration
tracker = Tracker(hw.grab_image, hw.get_position, settle=hw.settle) tracker = Tracker(hw.grab_image, hw.get_position, settle=hw.settle)
direction_array: np.ndarray = np.array(direction) direction_array: np.ndarray = np.array(direction)
@ -215,7 +238,7 @@ class CameraStageMapper(lt.Thing):
def calibrate_xy( def calibrate_xy(
self, hw: HardwareInterfaceDep, stage: Stage, logger: lt.deps.InvocationLogger self, hw: HardwareInterfaceDep, stage: Stage, logger: lt.deps.InvocationLogger
) -> DenumpifyingDict: ) -> DenumpifyingDict:
"""Move the microscope's stage in X and Y, to calibrate its relationship to the camera """Move the microscope's stage in X and Y, to calibrate its relationship to the camera.
This performs two 1d calibrations in x and y, then combines their results. This performs two 1d calibrations in x and y, then combines their results.
""" """
@ -265,14 +288,16 @@ class CameraStageMapper(lt.Thing):
that conversion. that conversion.
It is often helpful to give a concrete example: to make a move in image coordinates It is often helpful to give a concrete example: to make a move in image coordinates
(`dy`, `dx`), where `dx` is horizontal, i.e. the longer dimension of the image, you (``dy``, ``dx``), where ``dx`` is horizontal, i.e. the longer dimension of the image, you
should move the stage by: should move the stage by:
```
stage_disp = np.dot( .. code-block:: python
np.array(image_to_stage_displacement_matrix),
np.array([dy,dx]), stage_disp = np.dot(
) np.array(image_to_stage_displacement_matrix),
``` np.array([dy,dx]),
)
""" """
if self.last_calibration is None: if self.last_calibration is None:
return None return None
@ -282,21 +307,19 @@ class CameraStageMapper(lt.Thing):
return np.array(displacement_matrix).tolist() return np.array(displacement_matrix).tolist()
last_calibration = lt.ThingSetting( last_calibration = lt.ThingSetting(
initial_value=None, initial_value=None, model=Optional[dict], readonly=True
model=Optional[dict],
readonly=True,
description="The most recent CSM calibration",
) )
"""The most recent CSM calibration."""
@lt.thing_property @lt.thing_property
def image_resolution(self) -> Optional[Tuple[float, float]]: def image_resolution(self) -> Optional[Tuple[float, float]]:
"""The image size used to calibrate the image_to_stage_displacement_matrix""" """The image size used to calibrate the image_to_stage_displacement_matrix."""
if self.last_calibration is None: if self.last_calibration is None:
return None return None
return self.last_calibration["image_resolution"] return self.last_calibration["image_resolution"]
def assert_calibrated(self): def assert_calibrated(self):
"""Raise an exception if the image_to_stage_displacement matrix is not set""" """Raise an exception if the image_to_stage_displacement matrix is not set."""
if self.image_to_stage_displacement_matrix is None: if self.image_to_stage_displacement_matrix is None:
# Disable check of no message in raised exception as the message is explicitly # Disable check of no message in raised exception as the message is explicitly
# added by CSMUncalibratedError # added by CSMUncalibratedError
@ -309,15 +332,15 @@ class CameraStageMapper(lt.Thing):
x: float, x: float,
y: float, y: float,
): ):
"""Move by a given number of pixels on the camera """Move by a given number of pixels on the camera.
NB x and y here refer to what is usually understood to be the horizontal and NB x and y here refer to what is usually understood to be the horizontal and
vertical axes of the image. In many toolkits, "matrix indices" are used, which vertical axes of the image. In many toolkits, "matrix indices" are used, which
swap the order of these coordinates. This includes opencv and PIL. So, don't be swap the order of these coordinates. This includes opencv and PIL. So, don't be
surprised if you find it necessary to swap x and y around. surprised if you find it necessary to swap x and y around.
As a general rule, `x` usually corresponds to the longer dimension of the image, As a general rule, ``x`` usually corresponds to the longer dimension of the image,
and `y` to the shorter one. Checking what shape your chosen toolkit reports for and ``y`` to the shorter one. Checking what shape your chosen toolkit reports for
an image usually helps resolve any ambiguity. an image usually helps resolve any ambiguity.
""" """
self.assert_calibrated() self.assert_calibrated()
@ -328,7 +351,7 @@ class CameraStageMapper(lt.Thing):
@lt.thing_property @lt.thing_property
def thing_state(self) -> dict[str, Any]: def thing_state(self) -> dict[str, Any]:
"""Summary metadata describing the current state of the Thing""" """Summary metadata describing the current state of the Thing."""
return { return {
k: getattr(self, k) k: getattr(self, k)
for k in ["image_to_stage_displacement_matrix", "image_resolution"] for k in ["image_to_stage_displacement_matrix", "image_resolution"]

33
src/openflexure_microscope_server/things/settings_manager.py
View file

@ -1,5 +1,4 @@
""" """OpenFlexure Settings Management.
OpenFlexure Settings Management
This module provides some settings management across the other Things, and This module provides some settings management across the other Things, and
for code that currently lives in clients but needs to persist settings on for code that currently lives in clients but needs to persist settings on
@ -15,6 +14,7 @@ import labthings_fastapi as lt
def thing_server_from_request(request: Request) -> lt.ThingServer: def thing_server_from_request(request: Request) -> lt.ThingServer:
"""Wrap lt.find_thing_server."""
return lt.find_thing_server(request.app) return lt.find_thing_server(request.app)
@ -22,7 +22,7 @@ ThingServerDep = Annotated[lt.ThingServer, Depends(thing_server_from_request)]
def recursive_update(old_dict: MutableMapping, update: Mapping): def recursive_update(old_dict: MutableMapping, update: Mapping):
"""Update a dictionary recursively""" """Update a dictionary recursively."""
for k, v in update.items(): for k, v in update.items():
if isinstance(v, Mapping): if isinstance(v, Mapping):
if k in old_dict and isinstance(old_dict[k], MutableMapping): if k in old_dict and isinstance(old_dict[k], MutableMapping):
@ -34,7 +34,7 @@ def recursive_update(old_dict: MutableMapping, update: Mapping):
def nested_dict_get(data: dict, key: Sequence[str], create=False) -> Any: def nested_dict_get(data: dict, key: Sequence[str], create=False) -> Any:
"""Index a dict-of-dicts with a sequence of strings""" """Index a dict-of-dicts with a sequence of strings."""
subdict = data subdict = data
for k in key: for k in key:
if k not in subdict and create: if k not in subdict and create:
@ -44,17 +44,23 @@ def nested_dict_get(data: dict, key: Sequence[str], create=False) -> Any:
class SettingsManager(lt.Thing): class SettingsManager(lt.Thing):
"""Provides functionality to other Things about the current server state.
The SettingsManager is used to get information the microscope ID, the hostname
and the state of other Things.
"""
external_metadata = lt.ThingSetting( external_metadata = lt.ThingSetting(
initial_value={}, initial_value={},
model=Mapping, model=Mapping,
description="External metadata stored in the server's settings",
) )
"""External metadata stored in the server's settings."""
external_metadata_in_state = lt.ThingSetting( external_metadata_in_state = lt.ThingSetting(
initial_value=[], initial_value=[],
model=Sequence[str], model=Sequence[str],
description='A list of strings that are included in the "state" metadata',
) )
"""Keys from ``external_metadata`` that are included in the "state" metadata."""
@lt.thing_action @lt.thing_action
def update_external_metadata( def update_external_metadata(
@ -66,9 +72,9 @@ class SettingsManager(lt.Thing):
recursively, i.e. if a key exists, it will be added to rather than recursively, i.e. if a key exists, it will be added to rather than
replaced. replaced.
If a key is supplied, we will treat `data` as being relative to that If a key is supplied, we will treat ``data`` as being relative to that
key, i.e. calling this action with `data={"a":1 }, key="foo/bar"` is key, i.e. calling this action with ``data={"a":1 }, key="foo/bar"`` is
equivalent to calling it with `data={"foo": {"bar": {"a": 1}}}`. equivalent to calling it with ``data={"foo": {"bar": {"a": 1}}}``.
""" """
metadata = self.external_metadata metadata = self.external_metadata
subdict = metadata subdict = metadata
@ -84,8 +90,8 @@ class SettingsManager(lt.Thing):
"""Delete a key from the stored metadata. """Delete a key from the stored metadata.
The key may contain forward slashes, which are understood to separate The key may contain forward slashes, which are understood to separate
levels of the dictionary - i.e. `'a/c'` will remove the `c` key from a levels of the dictionary - i.e. ``'a/c'`` will remove the ``c`` key from a
dictionary that looks like: `{'a': {'c': 1}, 'b': {'d': 2}}` dictionary that looks like: ``{'a': {'c': 1}, 'b': {'d': 2}}``
""" """
metadata = self.external_metadata metadata = self.external_metadata
try: try:
@ -101,7 +107,7 @@ class SettingsManager(lt.Thing):
@lt.thing_setting @lt.thing_setting
def microscope_id(self) -> UUID: def microscope_id(self) -> UUID:
"""A unique identifier for this microscope""" """A unique identifier for this microscope."""
if self._microscope_id is None: if self._microscope_id is None:
self._microscope_id = str(uuid4()) self._microscope_id = str(uuid4())
return UUID(self._microscope_id) return UUID(self._microscope_id)
@ -118,11 +124,12 @@ class SettingsManager(lt.Thing):
@lt.thing_action @lt.thing_action
def get_things_state(self, metadata_getter: lt.deps.GetThingStates) -> Mapping: def get_things_state(self, metadata_getter: lt.deps.GetThingStates) -> Mapping:
"""Metadata summarising the current state of all Things in the server""" """Metadata summarising the current state of all Things in the server."""
return metadata_getter() return metadata_getter()
@property @property
def thing_state(self) -> Mapping: def thing_state(self) -> Mapping:
"""Summary metadata describing the current state of the Thing."""
state = { state = {
"hostname": self.hostname, "hostname": self.hostname,
"microscope-uuid": str(self.microscope_id), "microscope-uuid": str(self.microscope_id),

180
src/openflexure_microscope_server/things/smart_scan.py
View file

@ -1,3 +1,12 @@
"""The core sample scanning functionality for the OpenFlexure Microscope.
SmartScan provides sample scanning functionality including automatic background
detection (via the `BackgroundDetectThing`) and automatic path planning via
`scan_planners`. It manages the directories of past scans via `scan_directories`.
It also controls external processes for live stitching composite images, and
the creation of the final stitched images.
"""
from typing import Optional, Mapping from typing import Optional, Mapping
import threading import threading
import os import os
@ -42,11 +51,12 @@ STITCHING_RESOLUTION = (820, 616)
class ScanNotRunningError(RuntimeError): class ScanNotRunningError(RuntimeError):
"""Exception called when scan not running that requires a scan to be running""" """Exception called when scan not running that requires a scan to be running."""
def _scan_running(method): def _scan_running(method):
""" """Decorate a method so that it will error if a scan is not running.
This decorator is used by all methods in SmartScanThing that are using This decorator is used by all methods in SmartScanThing that are using
the variables set for the scan. It will throw a runtime error if the variables set for the scan. It will throw a runtime error if
self._scan_logger is not set, as all scan variables are set at self._scan_logger is not set, as all scan variables are set at
@ -65,7 +75,20 @@ def _scan_running(method):
class SmartScanThing(lt.Thing): class SmartScanThing(lt.Thing):
def __init__(self, scans_folder): """A Thing for scanning samples and interacting with past scans.
SmartScanThing exposes all functionality for automatically scanning samples,
previewing live stitching, retrieving data from past scans, and for deleting
past scans.
"""
def __init__(self, scans_folder: str):
"""Initialise a SmartScanThing saving to and loading from the input directory.
:param scans_folder: This is the path to the directory where all scans will be
saved. Any scans already in this directory will be accessible through the
HTTP interface.
"""
self._scan_dir_manager = scan_directories.ScanDirectoryManager(scans_folder) self._scan_dir_manager = scan_directories.ScanDirectoryManager(scans_folder)
self._preview_stitch_popen = None self._preview_stitch_popen = None
self._preview_stitch_popen_lock = threading.Lock() self._preview_stitch_popen_lock = threading.Lock()
@ -114,7 +137,6 @@ class SmartScanThing(lt.Thing):
stopping once it is surrounded by "background" (as detected by the stopping once it is surrounded by "background" (as detected by the
background_detect Thing) or reaches the "max_range" measured in steps. background_detect Thing) or reaches the "max_range" measured in steps.
""" """
got_lock = self._scan_lock.acquire(timeout=0.1) got_lock = self._scan_lock.acquire(timeout=0.1)
if not got_lock: if not got_lock:
raise RuntimeError("Trying to run scan while scan is already running!") raise RuntimeError("Trying to run scan while scan is already running!")
@ -170,7 +192,7 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _check_background_and_csm_set(self): def _check_background_and_csm_set(self):
"""Before starting a scan, check that background and camera-stage-mapping are set """Before starting a scan, check that background and camera-stage-mapping are set.
Raise error if: Raise error if:
- background is to be skipped but is not set - background is to be skipped but is not set
@ -207,14 +229,14 @@ class SmartScanThing(lt.Thing):
def _move_to_next_point( def _move_to_next_point(
self, next_point: tuple[int, int], z_estimate: Optional[int] = None self, next_point: tuple[int, int], z_estimate: Optional[int] = None
) -> tuple[int, int, int]: ) -> tuple[int, int, int]:
"""Moves the stage to the next poistion. If no z_estimate is given then """Move the stage to the next position.
the current stage position is used. Must move to the estimated focused position
(although moving below would be marginally faster) because background detect is
most reliable at the focused position.
Returns the (x,y,z) with the chosen z_estimate If no z_estimate is given then the current stage position is used. Must move
to the estimated focused position (although moving below would be marginally
faster) because background detect is most reliable at the focused position.
:returns: the (x,y,z) with the chosen z_estimate
""" """
if z_estimate is None: if z_estimate is None:
z_estimate = self._stage.position["z"] z_estimate = self._stage.position["z"]
@ -229,13 +251,12 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _calc_displacement_from_test_image(self, overlap: int) -> tuple[int, int]: def _calc_displacement_from_test_image(self, overlap: int) -> tuple[int, int]:
""" """Take a test image and use camera stage mapping to calculate x and y displacement.
Take a test image and use camera stage mapping to calculate x and y displacement
:param overlap: The desired overlap as a fraction of the image. i.e. 0.5 means :param overlap: The desired overlap as a fraction of the image. i.e. 0.5 means
that each image should overlap its nearest neighbour by 50%. that each image should overlap its nearest neighbour by 50%.
Return (dx, dy) - the x and y displacments in steps :returns: (dx, dy) - the x and y displacements in steps
""" """
test_jpg = self._cam.grab_jpeg() test_jpg = self._cam.grab_jpeg()
test_image = np.array(Image.open(test_jpg.open())) test_image = np.array(Image.open(test_jpg.open()))
@ -268,9 +289,9 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _set_scan_data(self): def _set_scan_data(self):
""" """Set date for this scan to the ``self._scan_data`` variable.
This sets the self._scan_data dictionary. This needs to become a
dataclass. This needs to become a dataclass at some point.
""" """
overlap = self.overlap overlap = self.overlap
dx, dy = self._calc_displacement_from_test_image(overlap) dx, dy = self._calc_displacement_from_test_image(overlap)
@ -312,9 +333,9 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _save_scan_inputs_json(self): def _save_scan_inputs_json(self):
""" """Save scan inputs as a JSON file in the scan folder.
Save scan inputs as a JSON file in the scan folder, to allow
the user to review the settings used in the scan This file allows the user to review the settings used in the scan.
""" """
# Should this be a method of the scan_data dataclass? # Should this be a method of the scan_data dataclass?
@ -337,12 +358,10 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _update_scan_data_json(self, scan_result: str): def _update_scan_data_json(self, scan_result: str):
""" """Update scan data JSON file with data only known at the end of the scan.
Update scan data as a JSON file in the scan folder, with
data only known at the end of the scan.
Takes scan_result, a string that is either "success", Takes scan_result, a string that is either "success", "cancelled by user",
"cancelled by user", or the error that ended the scan. or the error that ended the scan.
""" """
# Should this be a method of the scan_data dataclass? # Should this be a method of the scan_data dataclass?
current_time = datetime.now().replace(microsecond=0) current_time = datetime.now().replace(microsecond=0)
@ -372,10 +391,7 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _manage_stitching_threads(self): def _manage_stitching_threads(self):
""" """Manage the stitching threads, starting them if needed and not already running."""
Manage the stitching threads, starting them if needed and not already running.
"""
# Assume 4 images means at least one offset in x and y, making the stitching # Assume 4 images means at least one offset in x and y, making the stitching
# well constrained. # well constrained.
if self._scan_images_taken > 3: if self._scan_images_taken > 3:
@ -384,13 +400,12 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _run_scan(self): def _run_scan(self):
""" """Prepare and run the main scan, and perform final actions on completion.
Prepare and run the main scan, handling the threads performing
stitching. Uses the result (or exception) from the scanning to
determine whether the scan should be stitched and the microscope
should return to the starting x,y,z position
"""
The result (or exception) from the main scan loop determines whether the
scan should be stitched and whether the microscope should return to the
starting x,y,z position.
"""
# Used to check if finally was reached via exception (except # Used to check if finally was reached via exception (except
# cancel by user) # cancel by user)
scan_successful = True scan_successful = True
@ -458,11 +473,11 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _main_scan_loop(self): def _main_scan_loop(self):
""" """Run the main loop of the scan.
The loop to run through during a scan, until no more scan x,y positions
This loop runs during a scan, until no more scan x,y positions
are remaining. are remaining.
""" """
# The initial plan for the scan should be a single x,y position. All future # The initial plan for the scan should be a single x,y position. All future
# moves will be planned around this point. In future, route planner could # moves will be planned around this point. In future, route planner could
# have multiple starting positions, each of which will be visited before the # have multiple starting positions, each of which will be visited before the
@ -473,7 +488,7 @@ class SmartScanThing(lt.Thing):
"max_dist": self._scan_data["max_dist"], "max_dist": self._scan_data["max_dist"],
} }
route_planner = scan_planners.SmartSpiral( route_planner = scan_planners.SmartSpiral(
intial_position=(self._stage.position["x"], self._stage.position["y"]), initial_position=(self._stage.position["x"], self._stage.position["y"]),
planner_settings=planner_settings, planner_settings=planner_settings,
) )
@ -527,7 +542,7 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _return_to_starting_position(self): def _return_to_starting_position(self):
"""Return to the initial scan position, if set""" """Return to the initial scan position, if set."""
self._scan_logger.info("Returning to starting position.") self._scan_logger.info("Returning to starting position.")
if self._starting_position is not None: if self._starting_position is not None:
self._stage.move_absolute( self._stage.move_absolute(
@ -536,8 +551,7 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _perform_final_stitch(self): def _perform_final_stitch(self):
"""Update the scan zip and perform final stitch of the data""" """Update the scan zip and perform final stitch of the data."""
if self._scan_images_taken <= 3: if self._scan_images_taken <= 3:
self._scan_logger.info("Not performing a stitch as 3 or fewer images taken") self._scan_logger.info("Not performing a stitch as 3 or fewer images taken")
return return
@ -586,59 +600,54 @@ class SmartScanThing(lt.Thing):
save_resolution = lt.ThingSetting( save_resolution = lt.ThingSetting(
initial_value=(1640, 1232), initial_value=(1640, 1232),
model=tuple[int, int], model=tuple[int, int],
description=("A tuple of the image resolution to capture."),
) )
"""A tuple of the image resolution to capture."""
max_range = lt.ThingSetting( max_range = lt.ThingSetting(
initial_value=45000, initial_value=45000,
model=int, model=int,
description=(
"The maximum distance from the centre of the scan before we break in steps"
),
) )
"""The maximum distance in steps from the centre of the scan."""
stitch_tiff = lt.ThingSetting( stitch_tiff = lt.ThingSetting(
initial_value=False, initial_value=False,
model=bool, model=bool,
description="Whether or not to also produce a pyramidal tiff",
) )
"""Whether or not to also produce a pyramidal tiff at the end of a scan."""
skip_background = lt.ThingSetting( skip_background = lt.ThingSetting(
initial_value=True, initial_value=True,
model=bool, model=bool,
description="""Whether to detect and skip empty fields of view
This uses the settings from the `background_detect` Thing.""",
) )
"""Whether to detect and skip empty fields of view.
This uses the settings from the ``BackgroundDetectThing``."""
autofocus_dz = lt.ThingSetting( autofocus_dz = lt.ThingSetting(
initial_value=1000, initial_value=1000,
model=int, model=int,
description="The z distance to perform an autofocus in steps",
) )
"""The z distance to perform an autofocus in steps."""
overlap = lt.ThingSetting( overlap = lt.ThingSetting(
initial_value=0.45, initial_value=0.45,
model=float, model=float,
description="The fraction (0-1) that adjacent images should overlap in x or y",
) )
"""The fraction (0-1) that adjacent images should overlap in x or y."""
stitch_automatically = lt.ThingSetting( stitch_automatically = lt.ThingSetting(
initial_value=True, initial_value=True,
model=bool, model=bool,
description=(
"Whether to run a final stitch at the end of the scan (assuming scan "
"success)"
),
) )
"""Whether to run a final stitch at the end of a successful scan."""
@lt.thing_property @lt.thing_property
def scans(self) -> list[scan_directories.ScanInfo]: def scans(self) -> list[scan_directories.ScanInfo]:
"""All the available scans """All the available scans.
Each scan has a name (which can be used to access it), along with Each scan has a name (which can be used to access it), along with
its modified and created times (according to the filesystem) and its modified and created times (according to the filesystem) and
the number of items in the `images` folder. Note that image count the number of items in the ``images`` folder. Note that image count
uses a regular expression, and changes to the naming scheme will uses a regular expression, and changes to the naming scheme will
break it. break it.
""" """
@ -660,8 +669,8 @@ class SmartScanThing(lt.Thing):
"""Return the stitched image corresponding to a given scan name, if it exists. """Return the stitched image corresponding to a given scan name, if it exists.
Will only return a file ending in suffix STITCH_SUFFIX Will only return a file ending in suffix STITCH_SUFFIX
Note: when downloading this, the default filename will be `scan_name`.jpeg""" Note: when downloading this, the default filename will be ``scan_name``.jpeg
"""
stitch_path = self._scan_dir_manager.get_final_stitch_path(scan_name) stitch_path = self._scan_dir_manager.get_final_stitch_path(scan_name)
if stitch_path is None: if stitch_path is None:
@ -695,7 +704,7 @@ class SmartScanThing(lt.Thing):
"scans", "scans",
) )
def delete_all_scans(self, logger: lt.deps.InvocationLogger) -> None: def delete_all_scans(self, logger: lt.deps.InvocationLogger) -> None:
"""Delete all the scans on the microscope """Delete all the scans on the microscope.
**This will irreversibly remove all scanned data from the **This will irreversibly remove all scanned data from the
microscope!** microscope!**
@ -706,18 +715,17 @@ class SmartScanThing(lt.Thing):
@lt.thing_action @lt.thing_action
def purge_empty_scans(self, logger: lt.deps.InvocationLogger) -> None: def purge_empty_scans(self, logger: lt.deps.InvocationLogger) -> None:
""" """Delete all scan folders containing no images at the top level."""
Delete all scan folders containing no images at the top level
"""
# JSON is ignored as it's created before any images are captured # JSON is ignored as it's created before any images are captured
for scan_info in self._scan_dir_manager.all_scans_info(): for scan_info in self._scan_dir_manager.all_scans_info():
if scan_info.number_of_images == 0: if scan_info.number_of_images == 0:
self._delete_scan(scan_info.name, logger) self._delete_scan(scan_info.name, logger)
def _delete_scan(self, scan_name, logger: lt.deps.InvocationLogger) -> bool: def _delete_scan(self, scan_name, logger: lt.deps.InvocationLogger) -> bool:
""" """Delete a scan.
A wrapper around scan manager's delete_scan that logs to the invocation logger
This is a wrapper around scan manager's delete_scan that logs to the
invocation logger id there is a problem.
""" """
try: try:
self._scan_dir_manager.delete_scan(scan_name) self._scan_dir_manager.delete_scan(scan_name)
@ -731,8 +739,7 @@ class SmartScanThing(lt.Thing):
@property @property
def latest_preview_stitch_path(self) -> Optional[str]: def latest_preview_stitch_path(self) -> Optional[str]:
"""The path of the latest preview stitched image, or None if not available""" """The path of the latest preview stitched image, or None if not available."""
if not self.latest_scan_name: if not self.latest_scan_name:
return None return None
@ -742,11 +749,12 @@ class SmartScanThing(lt.Thing):
@lt.thing_property @lt.thing_property
def latest_preview_stitch_time(self) -> Optional[float]: def latest_preview_stitch_time(self) -> Optional[float]:
"""The modification time of the latest preview image, to allow live updating """The modification time of the latest preview image, to allow live updating.
This will return None (`null` to JS) if there is no preview image to return. This will return None (``null`` to JS) if there is no preview image to return.
This is used for two reasons: This is used for two reasons:
1. If all caching was turned off this stitch would be sent over the network 1. If all caching was turned off this stitch would be sent over the network
repeatedly repeatedly
2. If caching was is on, then the stitch will not update when needed. 2. If caching was is on, then the stitch will not update when needed.
@ -775,13 +783,12 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _preview_stitch_start(self, overlap: float) -> None: def _preview_stitch_start(self, overlap: float) -> None:
"""Start stitching a preview of the scan in a background subprocess """Start stitching a preview of the scan in a background subprocess.
This uses popen and returns immediately This uses popen and returns immediately
- self._preview_stitch_popen holds the popen for polling - self._preview_stitch_popen holds the popen for polling
- self._preview_stitch_popen_lock is a lock aquired while interacting - self._preview_stitch_popen_lock is a lock acquired while interacting
with Popen with Popen
""" """
# Set minimum overlap to 90% of the scan overlap to catch only images directly adjacent, # Set minimum overlap to 90% of the scan overlap to catch only images directly adjacent,
@ -805,7 +812,7 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _preview_stitch_running(self) -> bool: def _preview_stitch_running(self) -> bool:
"""Whether there is a preview stitch running in a subprocess""" """Whether there is a preview stitch running in a subprocess."""
with self._preview_stitch_popen_lock: with self._preview_stitch_popen_lock:
if self._preview_stitch_popen is None: if self._preview_stitch_popen is None:
return False return False
@ -815,9 +822,7 @@ class SmartScanThing(lt.Thing):
@_scan_running @_scan_running
def _preview_stitch_wait(self): def _preview_stitch_wait(self):
""" """Wait for an ongoing preview stitch to return."""
Wait for an ongoing preview stitch to return
"""
if self._preview_stitch_running(): if self._preview_stitch_running():
with self._preview_stitch_popen_lock: with self._preview_stitch_popen_lock:
self._preview_stitch_popen.wait() self._preview_stitch_popen.wait()
@ -828,18 +833,17 @@ class SmartScanThing(lt.Thing):
cancel: lt.deps.CancelHook, cancel: lt.deps.CancelHook,
cmd: list[str], cmd: list[str],
) -> CompletedProcess: ) -> CompletedProcess:
""" """Run a subprocess and log any output.
Run a subprocess and log any output
Raises: Raises:
ChildProcessError if exit code is not zero ChildProcessError if exit code is not zero
InvocationCancelledError if the action is cancelled. InvocationCancelledError if the action is cancelled.
""" """
logger.info(f"Running command in subprocess: `{' '.join(cmd)}`") logger.info(f"Running command in subprocess: `{' '.join(cmd)}`")
def log_buffer(buffer): def log_buffer(buffer):
"""A short internal function to read everything in the buffer to """Log everything in the buffer at INFO level."""
the log"""
while line := buffer.readline(): while line := buffer.readline():
logger.info(line) logger.info(line)
@ -852,7 +856,7 @@ class SmartScanThing(lt.Thing):
os.set_blocking(process.stdout.fileno(), False) os.set_blocking(process.stdout.fileno(), False)
logger.info(time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(time.time()))) logger.info(time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(time.time())))
# Poll returns None while running, will return the error code when finnished # Poll returns None while running, will return the error code when finished
while process.poll() is None: while process.poll() is None:
log_buffer(process.stdout) log_buffer(process.stdout)
# Once buffer is clear sleep for 0.2s before trying again. # Once buffer is clear sleep for 0.2s before trying again.
@ -883,7 +887,7 @@ class SmartScanThing(lt.Thing):
stitch_resize: Optional[float] = None, stitch_resize: Optional[float] = None,
overlap: float = 0.0, overlap: float = 0.0,
) -> None: ) -> None:
"""Generate a stitched image based on stage position metadata """Generate a stitched image based on stage position metadata.
Note that as this is a lt.thing_action it needs the logger passed as Note that as this is a lt.thing_action it needs the logger passed as
a variable if called from another thing action a variable if called from another thing action
@ -967,7 +971,9 @@ class SmartScanThing(lt.Thing):
self, self,
scan_name: str, scan_name: str,
): ):
"""Update the zip to include the files left until the end, then return the """Return zip after including any files left until the end.
zip file as a Blob"""
The zipfile is returned as a Blob.
"""
zip_fname = self._scan_dir_manager.zip_scan(scan_name, final_version=True) zip_fname = self._scan_dir_manager.zip_scan(scan_name, final_version=True)
return ZipBlob.from_file(zip_fname) return ZipBlob.from_file(zip_fname)

30
src/openflexure_microscope_server/things/stage/__init__.py
View file

@ -1,17 +1,27 @@
"""A package for stage control Things.
`BaseStage` is the base class that provides core stage functionality, but
no hardware interface control. To create a stage Thing to control a specific
piece of hardware the BaseStage should be subclassed, and any method raising
a NotImplementedError should be created.
As the object will be used as a context manager create the hardware connection in
``__enter__`` (not in ``__init__``), and close the connection with ``__exit__``.
"""
from __future__ import annotations from __future__ import annotations
from typing import TypeAlias
from collections.abc import Sequence, Mapping from collections.abc import Sequence, Mapping
import labthings_fastapi as lt import labthings_fastapi as lt
class BaseStage(lt.Thing): class BaseStage(lt.Thing):
"""A base stage class for OpenFlexure translation stages """A base stage class for OpenFlexure translation stages.
This can't be used directly but should reduce boilerplate code when This can't be used directly but should reduce boilerplate code when
implementing new stages. A minimal working stage must implement implementing new stages. A minimal working stage must implement
`move_relative` and `move_absolute` actions, which update the ``move_relative`` and ``move_absolute`` actions, which update the
`position` property on completion, and provide `set_zero_position`. ``position`` property on completion, and provide ``set_zero_position``.
""" """
_axis_names = ("x", "y", "z") _axis_names = ("x", "y", "z")
@ -24,22 +34,22 @@ class BaseStage(lt.Thing):
position = lt.ThingProperty( position = lt.ThingProperty(
Mapping[str, int], Mapping[str, int],
dict.fromkeys(_axis_names, 0), dict.fromkeys(_axis_names, 0),
description="Current position of the stage",
readonly=True, readonly=True,
observable=True, observable=True,
) )
"""Current position of the stage."""
moving = lt.ThingProperty( moving = lt.ThingProperty(
bool, bool,
False, False,
description="Whether the stage is in motion",
readonly=True, readonly=True,
observable=True, observable=True,
) )
"""Whether the stage is in motion."""
@property @property
def thing_state(self): def thing_state(self):
"""Summary metadata describing the current state of the stage""" """Summary metadata describing the current state of the stage."""
return {"position": self.position} return {"position": self.position}
@lt.thing_action @lt.thing_action
@ -68,7 +78,7 @@ class BaseStage(lt.Thing):
@lt.thing_action @lt.thing_action
def set_zero_position(self): def set_zero_position(self):
"""Make the current position zero in all axes """Make the current position zero in all axes.
This action does not move the stage, but resets the position to zero. This action does not move the stage, but resets the position to zero.
It is intended for use after manually or automatically recentring the It is intended for use after manually or automatically recentring the
@ -79,6 +89,4 @@ class BaseStage(lt.Thing):
) )
StageDependency: TypeAlias = lt.deps.direct_thing_client_dependency( StageDependency = lt.deps.direct_thing_client_dependency(BaseStage, "/stage/")
BaseStage, "/stage/"
)

16
src/openflexure_microscope_server/things/stage/dummy.py
View file

@ -1,3 +1,5 @@
"""Functionality for mimicking a stage during simulation and testing."""
from __future__ import annotations from __future__ import annotations
from collections.abc import Mapping from collections.abc import Mapping
@ -9,21 +11,29 @@ from . import BaseStage
class DummyStage(BaseStage): class DummyStage(BaseStage):
"""A dummy stage for testing purposes """A dummy stage for testing purposes.
This stage should work similarly to a Sangaboard stage, but without any This stage should work similarly to a Sangaboard stage, but without any
hardware attached. hardware attached.
""" """
def __init__(self, step_time: float = 0.001, **kwargs): def __init__(self, step_time: float = 0.001, **kwargs):
"""Initialise the Dummy stage, setting the step_time to adjust the speed.
:param step_time: The time in seconds per "motor" step. The default of 0.001
works well for the live simulation. For unit testing it is very slow
so the speed can be increased. Increasing it too far is problematic if
also doing computationally heavy tasks like simulated image blurring.
"""
super().__init__(**kwargs) super().__init__(**kwargs)
self.step_time = step_time self.step_time = step_time
def __enter__(self): def __enter__(self):
"""Register the stage position when the Thing context manager is opened."""
self.instantaneous_position = self.position self.instantaneous_position = self.position
def __exit__(self, _exc_type, _exc_value, _traceback): def __exit__(self, _exc_type, _exc_value, _traceback):
pass """Nothing to do when the Thing context manager is closed."""
@lt.thing_action @lt.thing_action
def move_relative( def move_relative(
@ -83,7 +93,7 @@ class DummyStage(BaseStage):
@lt.thing_action @lt.thing_action
def set_zero_position(self): def set_zero_position(self):
"""Make the current position zero in all axes """Make the current position zero in all axes.
This action does not move the stage, but resets the position to zero. This action does not move the stage, but resets the position to zero.
It is intended for use after manually or automatically recentring the It is intended for use after manually or automatically recentring the

37
src/openflexure_microscope_server/things/stage/sangaboard.py
View file

@ -1,3 +1,5 @@
"""Provide a LabThings-FastAPI interface to the Sangaboard motor controller."""
from __future__ import annotations from __future__ import annotations
import logging import logging
import threading import threading
@ -13,18 +15,32 @@ from . import BaseStage
class SangaboardThing(BaseStage): class SangaboardThing(BaseStage):
def __init__(self, port: str = None, **kwargs): """A Thing to manage a Sangaboard motor controller.
"""A Thing to manage a Sangaboard motor controller
Internally, this uses the ``pysangaboard`` package from PyPi. This imports
as ``sangaboard``. As ``pysangaboard`` does not support some features added
to the Sangaboard firmware v1 (LED flashing, aborting moves, etc) this
functionality is accessed by directly querying the serial interface.
"""
def __init__(self, port: str = None, **kwargs):
"""Initialise SangaboardThing.
Initialise the "Thing", but do not initialise an underlying
``Sangaboard`` object from ``pysangaboard`` until the Thing context
manager is started.
:param port: The serial port for the Sangaboard. Optional, this is used
to stop the Sangaboard object querying available devices.
:param ``**kwargs``: Any other keyword arguments to be passed to the
Sangaboard class
Internally, this uses the `pysangaboard` package from PyPi. This imports
as `sangaboard`. As `pysangaboard` does not support some features added
to the Sangaboard firmware v1 (LED flashing, aborting moves, etc) this
functionality is accessed by directly querying the serial interface.
""" """
self.sangaboard_kwargs = kwargs self.sangaboard_kwargs = kwargs
self.sangaboard_kwargs["port"] = port self.sangaboard_kwargs["port"] = port
def __enter__(self): def __enter__(self):
"""Connect to the sangaboard when the Thing context manager is opened."""
self._sangaboard = sangaboard.Sangaboard(**self.sangaboard_kwargs) self._sangaboard = sangaboard.Sangaboard(**self.sangaboard_kwargs)
self._sangaboard_lock = threading.RLock() self._sangaboard_lock = threading.RLock()
with self.sangaboard() as sb: with self.sangaboard() as sb:
@ -36,14 +52,15 @@ class SangaboardThing(BaseStage):
self.update_position() self.update_position()
def __exit__(self, _exc_type, _exc_value, _traceback): def __exit__(self, _exc_type, _exc_value, _traceback):
"""Close the sangaboard connection when the Thing context manager is closed."""
with self.sangaboard() as sb: with self.sangaboard() as sb:
sb.close() sb.close()
@contextmanager @contextmanager
def sangaboard(self) -> Iterator[sangaboard.Sangaboard]: def sangaboard(self) -> Iterator[sangaboard.Sangaboard]:
"""Return the wrapped `sangaboard.Sangaboard` instance. """Return the wrapped ``sangaboard.Sangaboard`` instance.
This is protected by a `threading.RLock`, which may change in future. This is protected by a ``threading.RLock``, which may change in future.
""" """
with self._sangaboard_lock: with self._sangaboard_lock:
yield self._sangaboard yield self._sangaboard
@ -102,7 +119,7 @@ class SangaboardThing(BaseStage):
@lt.thing_action @lt.thing_action
def set_zero_position(self) -> None: def set_zero_position(self) -> None:
"""Make the current position zero in all axes """Make the current position zero in all axes.
This action does not move the stage, but resets the position to zero. This action does not move the stage, but resets the position to zero.
It is intended for use after manually or automatically recentring the It is intended for use after manually or automatically recentring the
@ -119,7 +136,7 @@ class SangaboardThing(BaseStage):
dt: float = 0.5, dt: float = 0.5,
led_channel: Literal["cc"] = "cc", led_channel: Literal["cc"] = "cc",
) -> None: ) -> None:
"""Flash the LED to identify the board """Flash the LED to identify the board.
This is intended to be useful in situations where there are multiple This is intended to be useful in situations where there are multiple
Sangaboards in use, and it is necessary to identify which one is Sangaboards in use, and it is necessary to identify which one is

19
src/openflexure_microscope_server/things/system_control.py
View file

@ -1,5 +1,4 @@
""" """OpenFlexure Microscope system control Thing.
OpenFlexure Microscope system control Thing
This module defines a Thing that can shut down or restart the host computer. This module defines a Thing that can shut down or restart the host computer.
""" """
@ -11,20 +10,18 @@ from pydantic import BaseModel
class CommandOutput(BaseModel): class CommandOutput(BaseModel):
"""A pydantic model passing the STDOUT and STDERR from a subprocess over HTTP."""
output: str output: str
error: str error: str
class SystemControlThing(lt.Thing): class SystemControlThing(lt.Thing):
""" """Attempt to shutdown the device."""
Attempt to shutdown the device
"""
@lt.thing_action @lt.thing_action
def shutdown(self) -> CommandOutput: def shutdown(self) -> CommandOutput:
""" """Attempt to shutdown the device."""
Attempt to shutdown the device
"""
p = subprocess.Popen( p = subprocess.Popen(
["sudo", "shutdown", "-h", "now"], ["sudo", "shutdown", "-h", "now"],
stderr=subprocess.PIPE, stderr=subprocess.PIPE,
@ -36,14 +33,12 @@ class SystemControlThing(lt.Thing):
@lt.thing_property @lt.thing_property
def is_raspberrypi() -> bool: def is_raspberrypi() -> bool:
""" """Return True if running on a Raspberry Pi."""
Checks if we are running on a Raspberry Pi.
"""
return os.path.exists("/usr/bin/raspi-config") return os.path.exists("/usr/bin/raspi-config")
@lt.thing_action @lt.thing_action
def reboot(self) -> CommandOutput: def reboot(self) -> CommandOutput:
"""Attempt to reboot the device""" """Attempt to reboot the device."""
p = subprocess.Popen( p = subprocess.Popen(
["sudo", "shutdown", "-r", "now"], ["sudo", "shutdown", "-r", "now"],
stderr=subprocess.PIPE, stderr=subprocess.PIPE,

43
src/openflexure_microscope_server/utilities.py
View file

@ -1,14 +1,12 @@
""" """Utility functions and classes."""
This module contains some utility functions and classes
"""
from threading import Thread from threading import Thread
class ErrorCapturingThread(Thread): class ErrorCapturingThread(Thread):
""" """Subclass of Thread that captures exceptions from the target function.
This is a a subclass or Thread. It wraps the target function in a
try-except block. It wraps the target function in a try-except block.
Execution will stop with an unhandled exception, but the exception will not be raised Execution will stop with an unhandled exception, but the exception will not be raised
until the join method is called. When the join method is called, the exception is until the join method is called. When the join method is called, the exception is
@ -19,9 +17,7 @@ class ErrorCapturingThread(Thread):
""" """
def __init__(self, group=None, target=None, args=None, kwargs=None, daemon=None): def __init__(self, group=None, target=None, args=None, kwargs=None, daemon=None):
""" """Initialise with the same arguments as Thread."""
Initialise with the same arguments as Thread
"""
# As all inputs are keywords we need to set the default values for args and kwargs: # As all inputs are keywords we need to set the default values for args and kwargs:
if args is None: if args is None:
args = () args = ()
@ -47,9 +43,10 @@ class ErrorCapturingThread(Thread):
) )
def join(self, timeout=None): def join(self, timeout=None):
""" """Join when the thread is complete.
Join when the thread is complete. If the thread ended due to an unhandled exception,
the exception will be raised when this method is called. If the thread ended due to an unhandled exception, the exception will be raised
when this method is called.
""" """
super().join(timeout) super().join(timeout)
# If there is an error in the error buffer clear the buffer and raise it # If there is an error in the error buffer clear the buffer and raise it
@ -60,20 +57,18 @@ class ErrorCapturingThread(Thread):
def _wrap_and_catch_errors(target, error_buffer, *args, **kwargs): def _wrap_and_catch_errors(target, error_buffer, *args, **kwargs):
""" """Run target function in a try-except block.
This function is designed only to be used by
ErrorCapturingThread
It will run a target function in a try-except block This function is designed only to be used by ErrorCapturingThread.
any errors caught are added to an empty list that
should be supplied as a keyword argument
Arguments: It will run a target function in a try-except block catching any exception.
* target - The target function to call If an exception is caught it is added to ``error_buffer``.
* error_buffer - An empty list that is used for returning
the exception from the thread :param target: The target function to call
*args: The arguments for the target function :param error_buffer: An empty list that is used for returning the exception from
**kwargs: The Keyword arguments for the target function the thread. It is a list to ensure it is passed by reference.
:param ``*args``: The arguments for the target function
:param ``**kwargs``: The Keyword arguments for the target function
""" """
try: try:
target(*args, **kwargs) target(*args, **kwargs)

6
tests/__init__.py
View file

@ -0,0 +1,6 @@
"""The unit-test suite for the OpenFlexure Microscope Server.
This package contains all of the unit tests that can be run without specific
hardware. See also the `hardware-specific-tests` directory and the
`integration-tests` directory for more testing!.
"""

5
tests/mock_things/__init__.py
View file

@ -0,0 +1,5 @@
"""A package of test module providing mock Things for testing.
The mock things do not subclass the original things to minimise the inflation of
test coverage.
"""

19
tests/mock_things/mock_autofocus.py
View file

@ -1,17 +1,24 @@
"""This testing submodule contains mock autofocus things. """Testing submodule with mock Autofocus Things.
These mocks are designed to be inserted as dependencies to give specific These mocks are designed to be inserted as dependencies to provide specific
functionality and returns. functionality and return values.
The mocks do not subclass, and instead return very specific defined answers The mocks do not subclass Things. Instead, they return predefined
to functions answers to functions.
""" """
class MockAutoFocusThing: class MockAutoFocusThing:
"""A mock autofocus Thing that imports no code from AutofocusThing.
The class needs functionality added to it over time as more complex
mocking is needed. It imports no code from AutofocusThing so that coverage
is not artificially inflated.
"""
# Counter for checking functions were called # Counter for checking functions were called
mock_call_count = {"looping_autofocus": 0} mock_call_count = {"looping_autofocus": 0}
def looping_autofocus(self, dz: int = 2000, start: str = "centre") -> None: # noqa: ARG002 def looping_autofocus(self, dz: int = 2000, start: str = "centre") -> None: # noqa: ARG002
"""This function mocks autofocus with no return""" """Mock autofocus with no return."""
self.mock_call_count["looping_autofocus"] += 1 self.mock_call_count["looping_autofocus"] += 1

19
tests/mock_things/mock_background_detect.py
View file

@ -1,16 +1,23 @@
"""This testing submodule contains mock background detect things. """Testing submodule with mock Background Detect Things.
These mocks are designed to be inserted as dependencies to give specific These mocks are designed to be inserted as dependencies to provide specific
functionality and returns. functionality and return values.
The mocks do not subclass, and instead return very specific defined answers The mocks do not subclass Things. Instead, they return predefined
to functions answers to functions.
""" """
from openflexure_microscope_server.things.background_detect import ChannelDistributions from openflexure_microscope_server.things.background_detect import ChannelDistributions
class MockBackgoundDetectThing: class MockBackgroundDetectThing:
"""A mock background detect Thing that imports no code from BackgroundDetectThing.
The class needs functionality added to it over time as more complex
mocking is needed. It imports no code from BackgroundDetectThing so that coverage
is not artificially inflated.
"""
background_distributions = ChannelDistributions( background_distributions = ChannelDistributions(
means=[128.0, 128.0, 128.0], means=[128.0, 128.0, 128.0],
standard_deviations=[3.0, 3.0, 3.0], standard_deviations=[3.0, 3.0, 3.0],

17
tests/mock_things/mock_csm.py
View file

@ -1,12 +1,19 @@
"""This testing submodule contains mock camera stage mapping things. """Testing submodule with mock Camera Stage Mapping Things.
These mocks are designed to be inserted as dependencies to give specific These mocks are designed to be inserted as dependencies to provide specific
functionality and returns. functionality and return values.
The mocks do not subclass, and instead return very specific defined answers The mocks do not subclass Things. Instead, they return predefined
to functions answers to functions.
""" """
class MockCSMThing: class MockCSMThing:
"""A mock CSM Thing that imports no code from CameraStageMapper.
The class needs functionality added to it over time as more complex
mocking is needed. It imports no code from CameraStageMapper so that coverage
is not artificially inflated.
"""
image_resolution = (123, 456) image_resolution = (123, 456)

17
tests/mock_things/mock_stage.py
View file

@ -1,12 +1,19 @@
"""This testing submodule contains stage things. """Testing submodule with mocks StageThings.
These mocks are designed to be inserted as dependencies to give specific These mocks are designed to be inserted as dependencies to provide specific
functionality and returns. functionality and return values.
The mocks do not subclass, and instead return very specific defined answers The mocks do not subclass existing Things. Instead, they return predefined
to functions answers to functions.
""" """
class MockStageThing: class MockStageThing:
"""A mock Thing for a stage that imports no code from BaseStage.
The class needs functionality added to it over time as more complex
mocking is needed. It imports no code from BaseStage so that coverage
is not artificially inflated.
"""
position = (111, 222, 333) position = (111, 222, 333)

30
tests/test_camera_buffer.py
View file

@ -1,6 +1,4 @@
""" """Tests for the built in buffer for the camera."""
Tests for the built in buffer for the camera.
"""
from random import randint from random import randint
@ -18,7 +16,7 @@ RANDOM_GENERATOR = np.random.default_rng()
def random_image(): def random_image():
"""Create a random image""" """Create a random image."""
imarray = RANDOM_GENERATOR.integers( imarray = RANDOM_GENERATOR.integers(
low=0, high=255, size=(100, 100, 3), dtype="uint8" low=0, high=255, size=(100, 100, 3), dtype="uint8"
) )
@ -26,14 +24,14 @@ def random_image():
def random_metadata(): def random_metadata():
"""Create a misc dictionary to pretend to be metadata""" """Create a misc dictionary to pretend to be metadata."""
# Not very metadata like, but we are just checking that the same dict it # Not very metadata like, but we are just checking that the same dict it
# is returned # is returned
return {"a": randint(1, 100), "b": randint(1, 100)} return {"a": randint(1, 100), "b": randint(1, 100)}
def test_add_and_get_image(): def test_add_and_get_image():
"""Check images can be captured and retrieved""" """Check images can be captured and retrieved."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
misc_image = random_image() misc_image = random_image()
buffer_id = mem_buf.add_image(misc_image) buffer_id = mem_buf.add_image(misc_image)
@ -46,7 +44,7 @@ def test_add_and_get_image():
def test_add_and_get_image_twice(): def test_add_and_get_image_twice():
"""Check images can be retrieved twice if remove flag set false""" """Check images can be retrieved twice if remove flag set false."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
misc_image = random_image() misc_image = random_image()
buffer_id = mem_buf.add_image(misc_image) buffer_id = mem_buf.add_image(misc_image)
@ -62,7 +60,7 @@ def test_add_and_get_image_twice():
def test_get_without_id(): def test_get_without_id():
"""Check images can be captured and retrieved without ID""" """Check images can be captured and retrieved without ID."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
misc_image = random_image() misc_image = random_image()
mem_buf.add_image(misc_image) mem_buf.add_image(misc_image)
@ -94,7 +92,7 @@ def test_get_two_images():
def test_get_two_images_without_setting_buffer_size(): def test_get_two_images_without_setting_buffer_size():
"""Check two images can't be retrieved if the buffer size isn't set""" """Check two images can't be retrieved if the buffer size isn't set."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
misc_image1 = random_image() misc_image1 = random_image()
misc_image2 = random_image() misc_image2 = random_image()
@ -108,14 +106,14 @@ def test_get_two_images_without_setting_buffer_size():
def test_buffer_size_changing(): def test_buffer_size_changing():
"""Check buffer size resets back to 1 when if not set""" """Check buffer size resets back to 1 when if not set."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
misc_image1 = random_image() misc_image1 = random_image()
misc_image2 = random_image() misc_image2 = random_image()
misc_image3 = random_image() misc_image3 = random_image()
buffer_id1 = mem_buf.add_image(misc_image1, buffer_max=3) buffer_id1 = mem_buf.add_image(misc_image1, buffer_max=3)
buffer_id2 = mem_buf.add_image(misc_image2, buffer_max=3) buffer_id2 = mem_buf.add_image(misc_image2, buffer_max=3)
# Third capture doen't set buffer size, so it will be reset # Third capture doesn't set buffer size, so it will be reset
buffer_id3 = mem_buf.add_image(misc_image3) buffer_id3 = mem_buf.add_image(misc_image3)
# As buffer size was reset, images 1 and 2 are deleted # As buffer size was reset, images 1 and 2 are deleted
with pytest.raises(NoImageInMemoryError): with pytest.raises(NoImageInMemoryError):
@ -128,7 +126,7 @@ def test_buffer_size_changing():
def test_capture_two_images_get_without_id(): def test_capture_two_images_get_without_id():
"""Check that all images are deleted when getting without id""" """Check that all images are deleted when getting without id."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
misc_image1 = random_image() misc_image1 = random_image()
misc_image2 = random_image() misc_image2 = random_image()
@ -144,7 +142,7 @@ def test_capture_two_images_get_without_id():
def test_buffer_size_respected(): def test_buffer_size_respected():
"""Capture 10 images with a buffer size of 5. Check only last 5 exist""" """Capture 10 images with a buffer size of 5. Check only last 5 exist."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
images = [] images = []
@ -165,7 +163,7 @@ def test_buffer_size_respected():
def test_clear_buffer(): def test_clear_buffer():
"""Capture 10 images clear the buffer and check they are gone""" """Capture 10 images clear the buffer and check they are gone."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
images = [] images = []
@ -186,7 +184,7 @@ def test_clear_buffer():
def test_get_metadata_too(): def test_get_metadata_too():
"""Capture 10 images with metadata and check metadata is returned as expected""" """Capture 10 images with metadata and check metadata is returned as expected."""
mem_buf = CameraMemoryBuffer() mem_buf = CameraMemoryBuffer()
images = [] images = []
@ -203,7 +201,7 @@ def test_get_metadata_too():
# Preallocate zipped data, to avoid long confusing lines # Preallocate zipped data, to avoid long confusing lines
zipped = zip(images, metadatas, buffer_ids) zipped = zip(images, metadatas, buffer_ids)
# Check both image and metdata # Check both image and metadata
for i, (image, metadata, buffer_id) in enumerate(zipped): for i, (image, metadata, buffer_id) in enumerate(zipped):
returned_image, returned_metadata = mem_buf.get_image(buffer_id) returned_image, returned_metadata = mem_buf.get_image(buffer_id)
assert image is returned_image assert image is returned_image

27
tests/test_dummy_server.py
View file

@ -1,3 +1,13 @@
"""Test the server without creating a full HTTP server and socket connection.
Rather than spinning up a full uvicorn webserver for each test these tests use
the FastAPI ``TestClient`` or to directly communicate with the underlying
LabThings-FastAPI code. This increases speed of testing significantly.
For tests that require a full running server see the ``integration-tests``
directory in the root of the repository.
"""
import json import json
import os import os
import tempfile import tempfile
@ -21,6 +31,7 @@ camera_stage_mapping.DEFAULT_SETTLING_TIME = 0 # skip the settling time for tes
@pytest.fixture @pytest.fixture
def thing_server(): def thing_server():
"""Yield a server with a very basic configuration."""
temp_folder = tempfile.TemporaryDirectory() temp_folder = tempfile.TemporaryDirectory()
server = lt.ThingServer(settings_folder=temp_folder.name) server = lt.ThingServer(settings_folder=temp_folder.name)
server.add_thing( server.add_thing(
@ -33,36 +44,46 @@ def thing_server():
server.add_thing(AutofocusThing(), "/autofocus/") server.add_thing(AutofocusThing(), "/autofocus/")
server.add_thing(CameraStageMapper(), "/camera_stage_mapping/") server.add_thing(CameraStageMapper(), "/camera_stage_mapping/")
assert os.path.exists(os.path.join(temp_folder.name, "camera/")) assert os.path.exists(os.path.join(temp_folder.name, "camera/"))
# NB yield is important: otherwise, the temp folder gets deleted before the test runs # Note: yield is important. If return is used the temp folder gets deleted
# before the test runs
yield server yield server
@pytest.fixture @pytest.fixture
def client(thing_server): def client(thing_server):
"""Yield a FastAPI TestClient for the server."""
with TestClient(thing_server.app) as client: with TestClient(thing_server.app) as client:
yield client yield client
@pytest.fixture @pytest.fixture
def slower_client(thing_server): def slower_client(thing_server):
"""Yield a FastAPI TestClient for the server with a slower moving stage.
The step time for the stage is 100 microseconds rather than
1 microsecond.
"""
thing_server.things["/stage/"].step_time = 0.0001 thing_server.things["/stage/"].step_time = 0.0001
with TestClient(thing_server.app) as client: with TestClient(thing_server.app) as client:
yield client yield client
def test_autofocus(slower_client): def test_autofocus(slower_client):
"""Test Fast Autofocus can run doesn't raise an exception."""
client = slower_client client = slower_client
autofocus = lt.ThingClient.from_url("/autofocus/", client) autofocus = lt.ThingClient.from_url("/autofocus/", client)
_ = autofocus.fast_autofocus() _ = autofocus.fast_autofocus()
def test_grab_jpeg(client): def test_grab_jpeg(client):
"""Check that grab_jpeg returns a blob that can be opened."""
camera = lt.ThingClient.from_url("/camera/", client) camera = lt.ThingClient.from_url("/camera/", client)
blob = camera.grab_jpeg() blob = camera.grab_jpeg()
_image = Image.open(blob.open()) _image = Image.open(blob.open())
def test_capture_jpeg_metadata(client): def test_capture_jpeg_metadata(client):
"""Check that the position is encoded into the image metadata."""
camera = lt.ThingClient.from_url("/camera/", client) camera = lt.ThingClient.from_url("/camera/", client)
blob = camera.capture_jpeg() blob = camera.capture_jpeg()
image = Image.open(blob.open()) image = Image.open(blob.open())
@ -73,6 +94,7 @@ def test_capture_jpeg_metadata(client):
def test_stage(client): def test_stage(client):
"""Test moving th stage forwards and backwards."""
stage = lt.ThingClient.from_url("/stage/", client) stage = lt.ThingClient.from_url("/stage/", client)
start = stage.position start = stage.position
move = {"x": 1, "y": 2, "z": 3} move = {"x": 1, "y": 2, "z": 3}
@ -87,12 +109,13 @@ def test_stage(client):
def test_capture_array(client): def test_capture_array(client):
"""Capture array from simulation and check the size is as expected."""
camera = lt.ThingClient.from_url("/camera/", client) camera = lt.ThingClient.from_url("/camera/", client)
array = np.asarray(camera.capture_array()) array = np.asarray(camera.capture_array())
assert array.shape == (240, 320, 3) assert array.shape == (240, 320, 3)
# Currently this fails, not yet sure why.
def test_camera_stage_mapping_calibration(client): def test_camera_stage_mapping_calibration(client):
"""Check that camera stage mapping can run without an exception."""
camera_stage_mapping = lt.ThingClient.from_url("/camera_stage_mapping/", client) camera_stage_mapping = lt.ThingClient.from_url("/camera_stage_mapping/", client)
camera_stage_mapping.calibrate_xy() camera_stage_mapping.calibrate_xy()

45
tests/test_scan_directories.py
View file

@ -1,3 +1,5 @@
"""Test the functionality in the scan_directories module."""
import tempfile import tempfile
import os import os
import math import math
@ -28,13 +30,13 @@ BASE_SCAN_DIR = os.path.join(tempfile.gettempdir(), "scans")
def _clear_scan_dir() -> None: def _clear_scan_dir() -> None:
"""Delete the scan dir""" """Delete the scan dir."""
if os.path.exists(BASE_SCAN_DIR): if os.path.exists(BASE_SCAN_DIR):
shutil.rmtree(BASE_SCAN_DIR) shutil.rmtree(BASE_SCAN_DIR)
def _add_fake_image(scan_dir: ScanDirectory) -> None: def _add_fake_image(scan_dir: ScanDirectory) -> None:
"""Make a fake image on disk in the scan directory""" """Make a fake image on disk in the scan directory."""
unique = False unique = False
while not unique: while not unique:
x_pos = random.randint(-10000, 10000) x_pos = random.randint(-10000, 10000)
@ -63,12 +65,11 @@ def _add_fake_file(
def _make_fake_dzi(scan_dir: ScanDirectory, n_layers: int = 8) -> None: def _make_fake_dzi(scan_dir: ScanDirectory, n_layers: int = 8) -> None:
"""Create a fake DZI in a scan """Create a fake DZI in a scan.
:param n_layers: The number of layers of tiles. I.e. tile directories numbered :param n_layers: The number of layers of tiles. I.e. tile directories numbered
0...(n_layers-1) will be created. Default 8 0...(n_layers-1) will be created. Default 8
""" """
# Add an a dzi image # Add an a dzi image
dzi_fname = scan_dir.name + ".dzi" dzi_fname = scan_dir.name + ".dzi"
dzi_path = os.path.join(scan_dir.images_dir, dzi_fname) dzi_path = os.path.join(scan_dir.images_dir, dzi_fname)
@ -97,7 +98,7 @@ def _make_fake_dzi(scan_dir: ScanDirectory, n_layers: int = 8) -> None:
def test_basic_directory_operations(): def test_basic_directory_operations():
"""Test some basic operations """Test some basic operations.
Test some basic operations, including: Test some basic operations, including:
- ScanDirectoryManager creates a scan directory - ScanDirectoryManager creates a scan directory
@ -157,9 +158,7 @@ def test_basic_directory_operations():
def test_scan_sequence_and_listing(): def test_scan_sequence_and_listing():
""" """Check created scans are added in order and listed correctly."""
Check created scans are added in order and listed correctly
"""
_clear_scan_dir() _clear_scan_dir()
scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR) scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR)
# Make 4 scans # Make 4 scans
@ -186,10 +185,7 @@ def test_scan_sequence_and_listing():
def test_scan_name_non_sequential(): def test_scan_name_non_sequential():
""" """Check new scan has the correct name if the directories are not sequential."""
Check created scans is the correct name if the directories
are not sequential
"""
_clear_scan_dir() _clear_scan_dir()
os.makedirs(os.path.join(BASE_SCAN_DIR, "fake_scan_0001")) os.makedirs(os.path.join(BASE_SCAN_DIR, "fake_scan_0001"))
os.makedirs(os.path.join(BASE_SCAN_DIR, "fake_scan_0002")) os.makedirs(os.path.join(BASE_SCAN_DIR, "fake_scan_0002"))
@ -214,9 +210,7 @@ def test_scan_name_non_sequential():
def test_all_scan_names_taken(): def test_all_scan_names_taken():
""" """If the next sequential scan name needs more than 4 digits check error is thrown."""
If the next sequential scan name needs more than 4 digits check error is thrown.
"""
_clear_scan_dir() _clear_scan_dir()
os.makedirs(os.path.join(BASE_SCAN_DIR, "fake_scan_9999")) os.makedirs(os.path.join(BASE_SCAN_DIR, "fake_scan_9999"))
scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR) scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR)
@ -225,9 +219,7 @@ def test_all_scan_names_taken():
def test_no_scan_names_given(): def test_no_scan_names_given():
""" """Check correct default scan name is used if empty string is given."""
Check correct default scan name is used if empty string is given
"""
_clear_scan_dir() _clear_scan_dir()
scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR) scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR)
scan_dir = scan_dir_manager.new_scan_dir("") scan_dir = scan_dir_manager.new_scan_dir("")
@ -235,7 +227,7 @@ def test_no_scan_names_given():
def test_scan_info(): def test_scan_info():
"""Test the scan info is correct even using fake scan data""" """Test the scan info is correct even using fake scan data."""
_clear_scan_dir() _clear_scan_dir()
scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR) scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR)
scan_dir = scan_dir_manager.new_scan_dir("fake_scan") scan_dir = scan_dir_manager.new_scan_dir("fake_scan")
@ -262,7 +254,7 @@ def test_scan_info():
def test_get_final_stitch(): def test_get_final_stitch():
"""Check that the final stitch can be retrieved""" """Check that the final stitch can be retrieved."""
_clear_scan_dir() _clear_scan_dir()
scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR) scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR)
scan_dir = scan_dir_manager.new_scan_dir("fake_scan") scan_dir = scan_dir_manager.new_scan_dir("fake_scan")
@ -290,7 +282,7 @@ def test_get_final_stitch():
def test_empty_scan_info(): def test_empty_scan_info():
"""Test the scan info is correct even if the scan is empty""" """Test the scan info is correct even if the scan is empty."""
_clear_scan_dir() _clear_scan_dir()
scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR) scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR)
scan_dir = scan_dir_manager.new_scan_dir("fake_scan") scan_dir = scan_dir_manager.new_scan_dir("fake_scan")
@ -308,8 +300,7 @@ def test_empty_scan_info():
def test_zipping_scan_data(): def test_zipping_scan_data():
"""Test zipping the scan images with fake image data""" """Test zipping the scan images with fake image data."""
# Run twice, once calling the ScanDirectory directly, # Run twice, once calling the ScanDirectory directly,
# Once calling the ScanDirectoryManager # Once calling the ScanDirectoryManager
for caller in ["scan_dir", "manager"]: for caller in ["scan_dir", "manager"]:
@ -349,7 +340,7 @@ def test_zipping_scan_data():
def test_all_files(): def test_all_files():
"""Test all_files returns the path, and respects skipped directories""" """Test all_files returns the path, and respects skipped directories."""
_clear_scan_dir() _clear_scan_dir()
scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR) scan_dir_manager = ScanDirectoryManager(BASE_SCAN_DIR)
scan_dir = scan_dir_manager.new_scan_dir("fake_scan") scan_dir = scan_dir_manager.new_scan_dir("fake_scan")
@ -404,7 +395,7 @@ def test_none_returned_for_missing_images_dir():
def test_extracting_files(): def test_extracting_files():
"""Test the private _find_files method of ScanDirectories """Test the private _find_files method of ScanDirectories.
Add files to directory and check expected returns. Add files to directory and check expected returns.
""" """
@ -456,7 +447,7 @@ DiskUsage = namedtuple("DiskUsage", ["total", "used", "free"])
@pytest.mark.parametrize("free_space", [500_000_001, 700_000_000, 5_000_000_000]) @pytest.mark.parametrize("free_space", [500_000_001, 700_000_000, 5_000_000_000])
def test_disk_not_full(mocker, free_space): def test_disk_not_full(mocker, free_space):
"""Check no error thrown if disk has over 500MB of space""" """Check no error thrown if disk has over 500MB of space."""
total_space = 16_000_000_000 total_space = 16_000_000_000
# Mock the disk_usage # Mock the disk_usage
mocker.patch( mocker.patch(
@ -472,7 +463,7 @@ def test_disk_not_full(mocker, free_space):
@pytest.mark.parametrize("free_space", [100_000_000, 499_999_999]) @pytest.mark.parametrize("free_space", [100_000_000, 499_999_999])
def test_disk_full(mocker, free_space): def test_disk_full(mocker, free_space):
"""Check error thrown if disk has under 500MB of space""" """Check error thrown if disk has under 500MB of space."""
total_space = 16_000_000_000 total_space = 16_000_000_000
# Mock the disk_usage # Mock the disk_usage
mocker.patch( mocker.patch(

92
tests/test_scan_planners.py
View file

@ -1,3 +1,9 @@
"""Test the scan planning algorithms of the Microscope.
As well as low level function by function tests, this test suite also provides tests
that simulate scanning a sample, checking that the expected path is followed.
"""
import pytest import pytest
from copy import copy from copy import copy
@ -6,6 +12,7 @@ from .utilities import scan_test_helpers
def test_enforce_xy_tuple(): def test_enforce_xy_tuple():
"""Check that 2 value tuples (or ValueErrors) are always returned."""
bad_len_vals = [[1], [], (1,), (2, 4, 4), [1, 4, 5, 7]] bad_len_vals = [[1], [], (1,), (2, 4, 4), [1, 4, 5, 7]]
bad_type_vals = ["hi", 1, {"this": "that"}, {1, 2}] bad_type_vals = ["hi", 1, {"this": "that"}, {1, 2}]
for value in bad_len_vals + bad_type_vals: for value in bad_len_vals + bad_type_vals:
@ -17,6 +24,7 @@ def test_enforce_xy_tuple():
def test_enforce_xyz_tuple(): def test_enforce_xyz_tuple():
"""Check that 3 value tuples (or ValueErrors) are always returned."""
bad_len_vals = [[1], [], (1,), (2, 4), [1, 4, 5, 7]] bad_len_vals = [[1], [], (1,), (2, 4), [1, 4, 5, 7]]
bad_type_vals = ["hi!", 1, {"this": "that"}, {1, 2, 3}] bad_type_vals = ["hi!", 1, {"this": "that"}, {1, 2, 3}]
for value in bad_len_vals + bad_type_vals: for value in bad_len_vals + bad_type_vals:
@ -28,23 +36,25 @@ def test_enforce_xyz_tuple():
def test_base_class_not_implemented(): def test_base_class_not_implemented():
intial_position = (100, 50) """Check NotImplementedError is raised when initialising ScanPlanner directly."""
initial_position = (100, 50)
with pytest.raises(NotImplementedError): with pytest.raises(NotImplementedError):
scan_planners.ScanPlanner(intial_position=intial_position) scan_planners.ScanPlanner(initial_position=initial_position)
def test_v_basic_smart_spiral(): def test_v_basic_smart_spiral():
intial_position = (100, 50) """Check that a SmartSpiral where the first image is not sample completes."""
initial_position = (100, 50)
planner_settings = {"dx": 50, "dy": 50, "max_dist": 10000} planner_settings = {"dx": 50, "dy": 50, "max_dist": 10000}
planner = scan_planners.SmartSpiral( planner = scan_planners.SmartSpiral(
intial_position=intial_position, planner_settings=planner_settings initial_position=initial_position, planner_settings=planner_settings
) )
# Create a planner. It shouldn't be complete. # Create a planner. It shouldn't be complete.
assert not planner.scan_complete assert not planner.scan_complete
# When we start it should want to stay in the inital pos and have # When we start it should want to stay in the initial pos and have
# no z_estimate # no z_estimate
xy_pos, z_pos = planner.get_next_location_and_z_estimate() xy_pos, z_pos = planner.get_next_location_and_z_estimate()
assert xy_pos == intial_position assert xy_pos == initial_position
assert z_pos is None assert z_pos is None
# Try to mark location as imaged with only xy_position # Try to mark location as imaged with only xy_position
@ -65,11 +75,12 @@ def test_v_basic_smart_spiral():
def test_bad_smart_spiral_settings(): def test_bad_smart_spiral_settings():
intial_position = (100, 50) """Check that KeyError is raised when SmartSpiral is given bad settings."""
initial_position = (100, 50)
# Class init should raise error if no planner_settings dictionary set # Class init should raise error if no planner_settings dictionary set
with pytest.raises(ValueError): with pytest.raises(ValueError):
scan_planners.SmartSpiral(intial_position=intial_position) scan_planners.SmartSpiral(initial_position=initial_position)
planner_settings = {"dx": 50, "dy": 50, "max_dist": 10000} planner_settings = {"dx": 50, "dy": 50, "max_dist": 10000}
keys = ["dx", "dy", "max_dist"] keys = ["dx", "dy", "max_dist"]
@ -80,7 +91,7 @@ def test_bad_smart_spiral_settings():
del bad_planner_settings[delkey] del bad_planner_settings[delkey]
with pytest.raises(KeyError): with pytest.raises(KeyError):
scan_planners.SmartSpiral( scan_planners.SmartSpiral(
intial_position=intial_position, planner_settings=bad_planner_settings initial_position=initial_position, planner_settings=bad_planner_settings
) )
# Class init should raise error if planner_settings if any value can't be cast # Class init should raise error if planner_settings if any value can't be cast
@ -91,29 +102,29 @@ def test_bad_smart_spiral_settings():
bad_planner_settings[badkey] = "I can't be converted to an int" bad_planner_settings[badkey] = "I can't be converted to an int"
with pytest.raises(ValueError): with pytest.raises(ValueError):
scan_planners.SmartSpiral( scan_planners.SmartSpiral(
intial_position=intial_position, planner_settings=bad_planner_settings initial_position=initial_position, planner_settings=bad_planner_settings
) )
def test_smart_spiral_first_few_pos(): def test_smart_spiral_first_few_pos():
""" """Test for correct data addition during initial scan positions.
This test is VERY long, not really a "unit". It checks step-by-step
that data is added correctly for the first few postions in a scan.
This should catch basic cases of if the algorithm is updated. This is a very long test, not strictly a "unit" test. It checks step by step
that data is added correctly for the first few positions in a scan. It is
intended to catch basic issues if the algorithm is updated.
""" """
intial_position = (100, 50) initial_position = (100, 50)
planner_settings = {"dx": 50, "dy": 50, "max_dist": 10000} planner_settings = {"dx": 50, "dy": 50, "max_dist": 10000}
# Create a planner # Create a planner
planner = scan_planners.SmartSpiral( planner = scan_planners.SmartSpiral(
intial_position=intial_position, planner_settings=planner_settings initial_position=initial_position, planner_settings=planner_settings
) )
# it shouldn't start complete # it shouldn't start complete
assert not planner.scan_complete assert not planner.scan_complete
# When we start it should want to stay in the inital pos and have # When we start it should want to stay in the initial pos and have
# no z_estimate # no z_estimate
xy_pos1, z_pos1 = planner.get_next_location_and_z_estimate() xy_pos1, z_pos1 = planner.get_next_location_and_z_estimate()
assert xy_pos1 == intial_position assert xy_pos1 == initial_position
assert z_pos1 is None assert z_pos1 is None
# Set a focus value # Set a focus value
z_focus = 10 z_focus = 10
@ -189,11 +200,12 @@ def test_smart_spiral_first_few_pos():
def test_smart_spiral_stops_on_max_dist(): def test_smart_spiral_stops_on_max_dist():
intial_position = (0, 0) """Test that if max distance is reached smart spiral really does stop."""
initial_position = (0, 0)
planner_settings = {"dx": 100, "dy": 100, "max_dist": 1000} planner_settings = {"dx": 100, "dy": 100, "max_dist": 1000}
# Create a planner # Create a planner
planner = scan_planners.SmartSpiral( planner = scan_planners.SmartSpiral(
intial_position=intial_position, planner_settings=planner_settings initial_position=initial_position, planner_settings=planner_settings
) )
while not planner.scan_complete: while not planner.scan_complete:
xy_pos, _ = planner.get_next_location_and_z_estimate() xy_pos, _ = planner.get_next_location_and_z_estimate()
@ -206,17 +218,12 @@ def test_smart_spiral_stops_on_max_dist():
def test_mark_wrong_location(): def test_mark_wrong_location():
""" """Check that an error is raised if a scan marks the wrong location as visited."""
This test is VERY long, not really a "unit". It checks step-by-step initial_position = (100, 50)
that data is added correctly for the first few postions in a scan.
This should catch basic caseses of if the algorithm is updated.
"""
intial_position = (100, 50)
planner_settings = {"dx": 50, "dy": 50, "max_dist": 10000} planner_settings = {"dx": 50, "dy": 50, "max_dist": 10000}
# Create a planner # Create a planner
planner = scan_planners.SmartSpiral( planner = scan_planners.SmartSpiral(
intial_position=intial_position, planner_settings=planner_settings initial_position=initial_position, planner_settings=planner_settings
) )
xy_pos, _ = planner.get_next_location_and_z_estimate() xy_pos, _ = planner.get_next_location_and_z_estimate()
@ -226,17 +233,18 @@ def test_mark_wrong_location():
planner.mark_location_visited(wrong_xyz_pos, imaged=True, focused=True) planner.mark_location_visited(wrong_xyz_pos, imaged=True, focused=True)
def test_closest_focus_wth_large_numbers(): def test_closest_focus_with_large_numbers():
"""Tests to check that everything works well with huge numbers of steps.
The number of steps gets very large on the micorscope. But most of the tests
above use smaller numbers for clarity.
""" """
The number of steps gets very large in reality runs some tests to check initial_position = (0, 0)
that everything works well with huge numbers of steps
"""
intial_position = (0, 0)
# Set this up, but we won't use the settings # Set this up, but we won't use the settings
planner_settings = {"dx": 10000, "dy": 10000, "max_dist": 100000} planner_settings = {"dx": 10000, "dy": 10000, "max_dist": 100000}
# Create a planner # Create a planner
planner = scan_planners.SmartSpiral( planner = scan_planners.SmartSpiral(
intial_position=intial_position, planner_settings=planner_settings initial_position=initial_position, planner_settings=planner_settings
) )
# Directly overwrite the private focussed locations list for test # Directly overwrite the private focussed locations list for test
@ -253,11 +261,19 @@ def test_closest_focus_wth_large_numbers():
def test_example_smart_spiral(): def test_example_smart_spiral():
"""Test the smart spiral scan algorithm on the sample types listed """Test the smart spiral scan algorithm on the different sample types.
below and defined in scan_test_helpers.load_sample_points
Will fail if the locations or path between locations visited has changed The sample types:
for any of the samples listed"""
* ``regular``
* ``lobed``
* ``core"``
These are defined in scan_test_helpers.load_sample_points
This will fail if the locations or path between locations visited has changed
for any of the samples listed.
"""
example_samples = [ example_samples = [
"regular", "regular",
"lobed", "lobed",

44
tests/test_smart_scan.py
View file

@ -1,5 +1,4 @@
""" """Test the SmartScanThing *without* connecting it to a LabThings Server.
Test the SmartScanThing *without* connecting it to a LabThings Server.
By testing without connecting to the LabThings server it is possible to By testing without connecting to the LabThings server it is possible to
directly poll any properties and to start any methods. directly poll any properties and to start any methods.
@ -31,7 +30,7 @@ from openflexure_microscope_server.things.smart_scan import (
from .mock_things.mock_csm import MockCSMThing from .mock_things.mock_csm import MockCSMThing
from .mock_things.mock_autofocus import MockAutoFocusThing from .mock_things.mock_autofocus import MockAutoFocusThing
from .mock_things.mock_stage import MockStageThing from .mock_things.mock_stage import MockStageThing
from .mock_things.mock_background_detect import MockBackgoundDetectThing from .mock_things.mock_background_detect import MockBackgroundDetectThing
# A global logger to pass in as an Invocation Logger # A global logger to pass in as an Invocation Logger
LOGGER = logging.getLogger("mock-invocation_logger") LOGGER = logging.getLogger("mock-invocation_logger")
@ -42,14 +41,14 @@ SCAN_DIR = os.path.join(tempfile.gettempdir(), "scans")
def _clear_scan_dir() -> None: def _clear_scan_dir() -> None:
"""Delete the scan dir""" """Delete the scan dir."""
if os.path.exists(SCAN_DIR): if os.path.exists(SCAN_DIR):
shutil.rmtree(SCAN_DIR) shutil.rmtree(SCAN_DIR)
@pytest.fixture @pytest.fixture
def smart_scan_thing(): def smart_scan_thing():
"""Return a smart scan thing as a fixture""" """Return a smart scan thing as a fixture."""
return SmartScanThing(SCAN_DIR) return SmartScanThing(SCAN_DIR)
@ -64,8 +63,11 @@ def test_initial_properties(smart_scan_thing):
def test_inaccessible_scan_methods(smart_scan_thing): def test_inaccessible_scan_methods(smart_scan_thing):
"""The @_scan_running decorator should make some methods """Test that method with @_scan_running decorator is inaccessible.
inaccessible unless a scan is running"""
The @_scan_running decorator makes these functions inaccessible unless
a scan is running.
"""
with pytest.raises(ScanNotRunningError): with pytest.raises(ScanNotRunningError):
smart_scan_thing._run_scan() smart_scan_thing._run_scan()
with pytest.raises(ScanNotRunningError): with pytest.raises(ScanNotRunningError):
@ -73,8 +75,7 @@ def test_inaccessible_scan_methods(smart_scan_thing):
def test_private_delete_scan(smart_scan_thing, caplog): def test_private_delete_scan(smart_scan_thing, caplog):
"""Test the private _delete_scan method deletes directories or warns if it can't""" """Test the private _delete_scan method deletes directories or warns if it can't."""
_clear_scan_dir() _clear_scan_dir()
with caplog.at_level(logging.INFO): with caplog.at_level(logging.INFO):
fake_scan_name = "fake_scan_0001" fake_scan_name = "fake_scan_0001"
@ -100,8 +101,7 @@ def test_private_delete_scan(smart_scan_thing, caplog):
def test_public_delete_scan(smart_scan_thing, caplog): def test_public_delete_scan(smart_scan_thing, caplog):
"""Test the delete_scan API call deletes directories or warns if it can't""" """Test the delete_scan API call deletes directories or warns if it can't."""
_clear_scan_dir() _clear_scan_dir()
with caplog.at_level(logging.INFO): with caplog.at_level(logging.INFO):
fake_scan_name = "fake_scan_0001" fake_scan_name = "fake_scan_0001"
@ -151,9 +151,8 @@ def test_delete_all_scans(smart_scan_thing, caplog):
assert len(caplog.records) == 0 assert len(caplog.records) == 0
def _run_only_outer_scan(adjust_inital_state: Optional[Callable] = None): def _run_only_outer_scan(adjust_initial_state: Optional[Callable] = None):
""" """Create a subclass of SmartScanThing to mock _run_scan and run sample_scan.
Create a subclass of SmartScanThing to mock _run_scan and run sample_scan
This should do all the set up for a scan, move into the mocked This should do all the set up for a scan, move into the mocked
_run_scan method where this can be tested. Once this is done _run_scan method where this can be tested. Once this is done
@ -166,7 +165,6 @@ def _run_only_outer_scan(adjust_inital_state: Optional[Callable] = None):
This seems hard to do with a fixture so it is being done with a private This seems hard to do with a fixture so it is being done with a private
function function
""" """
# cancel handle shouldn't be used. Set to arbitrary value for checking # cancel handle shouldn't be used. Set to arbitrary value for checking
cancel_mock = 1 # not called cancel_mock = 1 # not called
af_mock = MockAutoFocusThing() af_mock = MockAutoFocusThing()
@ -174,20 +172,14 @@ def _run_only_outer_scan(adjust_inital_state: Optional[Callable] = None):
cam_mock = 4 # not called cam_mock = 4 # not called
meta_mock = 5 # not called meta_mock = 5 # not called
csm_mock = MockCSMThing() csm_mock = MockCSMThing()
bkgrnd_det_mock = MockBackgoundDetectThing() bkgrnd_det_mock = MockBackgroundDetectThing()
class MockedSmartScanThing(SmartScanThing): class MockedSmartScanThing(SmartScanThing):
""" """Mocked version of SmartScanThing with a patched _run_scan method."""
This is a subclass of SmartScanThing with a mocked method and
mocked thing_settings.
"""
# Counter for checking functions were called # Counter for checking functions were called
mock_call_count = {"_run_scan": 0} mock_call_count = {"_run_scan": 0}
# Mock thing settings as a dictionary
thing_settings = {"skip_background": True}
def _run_scan(self): def _run_scan(self):
self.mock_call_count["_run_scan"] += 1 self.mock_call_count["_run_scan"] += 1
@ -207,8 +199,8 @@ def _run_only_outer_scan(adjust_inital_state: Optional[Callable] = None):
# mock smart scan thing # mock smart scan thing
mock_ss_thing = MockedSmartScanThing(SCAN_DIR) mock_ss_thing = MockedSmartScanThing(SCAN_DIR)
if adjust_inital_state is not None: if adjust_initial_state is not None:
adjust_inital_state(mock_ss_thing) adjust_initial_state(mock_ss_thing)
exec_info = None exec_info = None
try: try:
@ -240,7 +232,7 @@ def _run_only_outer_scan(adjust_inital_state: Optional[Callable] = None):
assert mock_ss_thing._scan_images_taken is None assert mock_ss_thing._scan_images_taken is None
# Return the mock thing for further state testing, and the # Return the mock thing for further state testing, and the
# exec_info of any uncaught exeptions that were raised # exec_info of any uncaught exceptions that were raised
return mock_ss_thing, exec_info return mock_ss_thing, exec_info

35
tests/test_stack.py
View file

@ -1,5 +1,4 @@
""" """Tests for the smart/fast stacking.
Tests for the smart/fast stacking.
Currently these tests don't test the Thing itself, just surrounding functionality Currently these tests don't test the Thing itself, just surrounding functionality
""" """
@ -23,7 +22,7 @@ RANDOM_GENERATOR = np.random.default_rng()
def odd_integers(min_value=0, max_value=1000): def odd_integers(min_value=0, max_value=1000):
"""A hypothesis strategy for odd integers""" """Return a hypothesis strategy for odd integers."""
min_base = (min_value) // 2 min_base = (min_value) // 2
max_base = (max_value - 1) // 2 max_base = (max_value - 1) // 2
# Ensure the range allows at least one odd number # Ensure the range allows at least one odd number
@ -33,7 +32,7 @@ def odd_integers(min_value=0, max_value=1000):
def even_integers(min_value=0, max_value=1000): def even_integers(min_value=0, max_value=1000):
"""A hypothesis strategy for even integers""" """Return a hypothesis strategy for even integers."""
min_base = (min_value + 1) // 2 min_base = (min_value + 1) // 2
max_base = (max_value) // 2 max_base = (max_value) // 2
# Ensure the range allows at least one even number # Ensure the range allows at least one even number
@ -47,14 +46,13 @@ def even_integers(min_value=0, max_value=1000):
extra_ims=even_integers(min_value=0, max_value=10), extra_ims=even_integers(min_value=0, max_value=10),
) )
def test_stack_params_validation(save_ims, extra_ims): def test_stack_params_validation(save_ims, extra_ims):
"""Tests specifically the validation on the image numbers """Test the validation of the image numbers for a stack.
save_ims is the number to save (must be odd and positive) save_ims is the number to save (must be odd and positive)
extra_ims is how many more images there are in min_images_to_test than extra_ims is how many more images there are in min_images_to_test than
images_to_save. (even so that, min_images_to_test is odd and larger than images_to_save. (even so that, min_images_to_test is odd and larger than
images_to_save images_to_save
""" """
StackParams( StackParams(
stack_dz=50, stack_dz=50,
images_to_save=save_ims, images_to_save=save_ims,
@ -70,8 +68,9 @@ def test_stack_params_validation(save_ims, extra_ims):
extra_ims=even_integers(min_value=-10, max_value=-1), extra_ims=even_integers(min_value=-10, max_value=-1),
) )
def test_stack_params_not_enough_test_images(save_ims, extra_ims): def test_stack_params_not_enough_test_images(save_ims, extra_ims):
"""Set the extra_ims negative so that min_images_to_test is smaller """Test error is raised if min_images_to_test is smaller than images_to_save.
than images_to_save.
``extra_ims`` negative so that min_images_to_test is smaller than images_to_save.
For arguments see test_stack_params_validation For arguments see test_stack_params_validation
""" """
@ -91,7 +90,7 @@ def test_stack_params_not_enough_test_images(save_ims, extra_ims):
extra_ims=even_integers(min_value=0, max_value=10), extra_ims=even_integers(min_value=0, max_value=10),
) )
def test_stack_params_negative_images_to_save(save_ims, extra_ims): def test_stack_params_negative_images_to_save(save_ims, extra_ims):
"""save_ims is negative so images_to_save is negative, failing validation """save_ims is negative so images_to_save is negative, failing validation.
For arguments see test_stack_params_validation For arguments see test_stack_params_validation
""" """
@ -111,7 +110,7 @@ def test_stack_params_negative_images_to_save(save_ims, extra_ims):
extra_ims=odd_integers(min_value=0, max_value=10), extra_ims=odd_integers(min_value=0, max_value=10),
) )
def test_even_min_images_to_test(save_ims, extra_ims): def test_even_min_images_to_test(save_ims, extra_ims):
"""extra_ims is odd so min_images_to_test is even, failing validation """extra_ims is odd so min_images_to_test is even, failing validation.
For arguments see test_stack_params_validation For arguments see test_stack_params_validation
""" """
@ -131,7 +130,7 @@ def test_even_min_images_to_test(save_ims, extra_ims):
extra_ims=odd_integers(min_value=0, max_value=10), extra_ims=odd_integers(min_value=0, max_value=10),
) )
def test_even_images_to_save(save_ims, extra_ims): def test_even_images_to_save(save_ims, extra_ims):
"""save_ims is even so images_to_save is even, failing validation """save_ims is even so images_to_save is even, failing validation.
For arguments see test_stack_params_validation For arguments see test_stack_params_validation
""" """
@ -147,8 +146,7 @@ def test_even_images_to_save(save_ims, extra_ims):
def test_computed_stack_params(): def test_computed_stack_params():
""" """Test StackParams computed properties are as expected.
Test StackParams computed properties are as expected
Not using hypothesis or we will just copy in the same formulas. Not using hypothesis or we will just copy in the same formulas.
""" """
@ -189,8 +187,7 @@ def test_computed_stack_params():
def random_capture(set_id: Optional[int] = None): def random_capture(set_id: Optional[int] = None):
""" """Create a capture with random values.
Create a capture with random values
:param set_id: Optional, use to set a fixed id rather than a random one :param set_id: Optional, use to set a fixed id rather than a random one
""" """
@ -207,18 +204,14 @@ def random_capture(set_id: Optional[int] = None):
def test_capture_filename_matches_regex(): def test_capture_filename_matches_regex():
""" """For 100 random captures check the image always matches the regex."""
For 100 random captures check the image always matches the regex
"""
for _ in range(100): for _ in range(100):
assert IMAGE_REGEX.search(random_capture().filename) assert IMAGE_REGEX.search(random_capture().filename)
@given(st.integers(min_value=0, max_value=5000)) @given(st.integers(min_value=0, max_value=5000))
def test_retrieval_of_captures(start): def test_retrieval_of_captures(start):
""" """For 20 random captures, check each can be retrieved correctly by id."""
For 20 random captures, check each can be retrieved correctly by id
"""
captures = [random_capture(start + i) for i in range(20)] captures = [random_capture(start + i) for i in range(20)]
for i, capture in enumerate(captures): for i, capture in enumerate(captures):

12
tests/utilities/__init__.py
View file

@ -1,8 +1,7 @@
""" """Utilities for testing and debugging.
This sub-package contains utilities that help with testing and debugging.
At the top level are some very basic testing functions, more specific testing At the top level are some basic testing functions. More specific testing utilities are
is provided by modules inside the package. provided by modules inside the package.
""" """
from typing import Protocol, Iterable from typing import Protocol, Iterable
@ -10,9 +9,10 @@ from collections.abc import Hashable
class SizedIterableHashable(Iterable[Hashable], Protocol): class SizedIterableHashable(Iterable[Hashable], Protocol):
"""A protocol for sized iterable of hashable objects""" """A protocol for sized iterable of hashable objects."""
def __len__(self) -> int: ... def __len__(self) -> int:
"""Add a len function to protocol so Python knows the object is sized."""
def assert_unique_of_length(data: SizedIterableHashable, length: int) -> None: def assert_unique_of_length(data: SizedIterableHashable, length: int) -> None:

77
tests/utilities/scan_test_helpers.py
View file

@ -1,3 +1,9 @@
"""Utility functions for testing scan planners.
These including fake sample creation, scan path visualisation, and
persistent storage of expected scan paths for samples.
"""
import os import os
import pickle import pickle
@ -14,26 +20,23 @@ THIS_DIR = os.path.dirname(os.path.realpath(__file__))
class FakeSample: class FakeSample:
""" """A fake sample to test scan algorithms.
A fake sample to test scan algorithms. The sample is able to return
whether a given position is sample, no image associated with the sample The sample is able to return whether a given position is sample, there is no image
associated with the sample
""" """
def __init__(self, xy_points: list[tuple[int, int]]): def __init__(self, xy_points: list[tuple[int, int]]):
""" """Create the sample from a spline interpolation around the given points."""
Create the sample from a spline interpolation around
the given points.
"""
self._sample_perimeter = interp_closed_path(xy_points, 500) self._sample_perimeter = interp_closed_path(xy_points, 500)
def is_sample(self, pos: tuple[int, int], im_size: tuple[int, int]) -> bool: def is_sample(self, pos: tuple[int, int], im_size: tuple[int, int]) -> bool:
""" """Return True if an image at a given location is on the sample.
Return whether an image at a given location with a given image size
is on the sample
This doesn't check the entire image field as this is designed to be used The image size is specified to check if it overlaps the sample. It doesn't
where the fake sample is much larger than the image and has smooth edges check the entire image field as this is designed to be used where the fake
It just checks the 4 corners sample is much larger than the image and has smooth edges. It just checks the
4 corners.
""" """
img_corners = [ img_corners = [
(pos[0] + im_size[0], pos[1] + im_size[1]), (pos[0] + im_size[0], pos[1] + im_size[1]),
@ -47,18 +50,14 @@ class FakeSample:
@property @property
def patch(self) -> PathPatch: def patch(self) -> PathPatch:
""" """The sample as a matplotlib patch for plotting."""
The sample as a matplotlib patch for plotting
"""
patch = PathPatch(self._sample_perimeter) patch = PathPatch(self._sample_perimeter)
patch.set(color=(1.0, 0.8, 1.0, 1.0)) patch.set(color=(1.0, 0.8, 1.0, 1.0))
return patch return patch
def visualise_scan(sample: FakeSample, planner: scan_planners.ScanPlanner) -> Figure: def visualise_scan(sample: FakeSample, planner: scan_planners.ScanPlanner) -> Figure:
""" """For a given sample and scanner object return a matplotlib figure of the scan."""
For a given sample and scanner object return a matplotlib figure of the scan
"""
fig, ax = plt.subplots(figsize=(8, 8)) fig, ax = plt.subplots(figsize=(8, 8))
ax.add_artist(sample.patch) ax.add_artist(sample.patch)
xh, yh = zip(*planner._path_history) xh, yh = zip(*planner._path_history)
@ -84,15 +83,14 @@ def visualise_scan(sample: FakeSample, planner: scan_planners.ScanPlanner) -> Fi
def interp_closed_path(xy_points: list[tuple[int, int]], n_points: int) -> MatPath: def interp_closed_path(xy_points: list[tuple[int, int]], n_points: int) -> MatPath:
""" """Interpolate an n_point closed curve from a lists of xy_points.
Given a lists of xy_points interpolate an n_point closed curve. This can be used
to create an arbitrary sample shape plan a scan. This can be used to create an arbitrary sample shape for testing a scan planner.
Modified from: Modified from:
https://stackoverflow.com/questions/33962717/interpolating-a-closed-curve-using-scipy https://stackoverflow.com/questions/33962717/interpolating-a-closed-curve-using-scipy
""" """
# Use zip to separate x and y points into tuples
# Use zip to seperate x and y points into tuples
x, y = zip(*xy_points) x, y = zip(*xy_points)
# Append first point and convert to array # Append first point and convert to array
@ -114,17 +112,17 @@ def interp_closed_path(xy_points: list[tuple[int, int]], n_points: int) -> MatPa
def example_smart_spiral( def example_smart_spiral(
sample_name: str = "lobed", sample_name: str = "lobed",
) -> tuple[FakeSample, scan_planners.ScanPlanner]: ) -> tuple[FakeSample, scan_planners.ScanPlanner]:
""" """Run an example scan.
Run an example scan and return the sample scanned and the planner object
after scan is complete :returns: The sample scanned and the planner object after scan is complete.
""" """
xy_sample_points = load_sample_points(sample_name) xy_sample_points = load_sample_points(sample_name)
sample = FakeSample(xy_sample_points) sample = FakeSample(xy_sample_points)
img_size = (1000, 1000) img_size = (1000, 1000)
intial_position = (0, 0) initial_position = (0, 0)
planner_settings = {"dx": 1200, "dy": 800, "max_dist": 100000} planner_settings = {"dx": 1200, "dy": 800, "max_dist": 100000}
planner = scan_planners.SmartSpiral( planner = scan_planners.SmartSpiral(
intial_position=intial_position, planner_settings=planner_settings initial_position=initial_position, planner_settings=planner_settings
) )
while not planner.scan_complete: while not planner.scan_complete:
@ -136,12 +134,11 @@ def example_smart_spiral(
def profile_and_save_plot_for_example_smart_spiral(): def profile_and_save_plot_for_example_smart_spiral():
""" """Run the example scan and save a plot and the profile data.
Run the example scan and save a plot and the profile data
Also print the cumulative stats
Also print the cumulative stats.
This runs if you run this file directly This runs if you run this file directly.
""" """
import pstats import pstats
import cProfile import cProfile
@ -163,9 +160,9 @@ def profile_and_save_plot_for_example_smart_spiral():
def update_example_smart_spiral_pickle(sample_name: str): def update_example_smart_spiral_pickle(sample_name: str):
""" """Pickle the ScanPlanner for the example_smart_spiral().
Pickle the ScanPlanner for the example_smart_spiral(),
this is done so the history can be compared by testing to check This is done so the history can be compared by testing to check
the algorithm is unchanged. the algorithm is unchanged.
If the algorithm is purposefully changed then this will need to be If the algorithm is purposefully changed then this will need to be
@ -183,9 +180,9 @@ def update_example_smart_spiral_pickle(sample_name: str):
def get_expected_result_for_example_smart_spiral( def get_expected_result_for_example_smart_spiral(
sample_name: str, sample_name: str,
) -> scan_planners.ScanPlanner: ) -> scan_planners.ScanPlanner:
""" """Return the expected ScanPlanner object for the example_smart_spiral().
Return the expected ScanPlanner object for the example_smart_spiral(),
this is pickled, so that it can be committed. This is loaded from a pickle so that the object can be committed to the repo.
""" """
pkl_fname = os.path.join(THIS_DIR, f"example_smart_spiral_{sample_name}.pkl") pkl_fname = os.path.join(THIS_DIR, f"example_smart_spiral_{sample_name}.pkl")
with open(pkl_fname, "rb") as pkl_file_obj: with open(pkl_fname, "rb") as pkl_file_obj:
@ -193,7 +190,7 @@ def get_expected_result_for_example_smart_spiral(
def load_sample_points(sample_name: str): def load_sample_points(sample_name: str):
"""Return the points to generate the FakeSample corresponding to the given input name """Return the points to generate the FakeSample corresponding to the given input name.
Options are "lobed", "regular", and "core". Options are "lobed", "regular", and "core".
""" """

2
webapp/src/main.js
View file

@ -97,7 +97,7 @@ Vue.mixin({
} }
) )
.finally(function() { .finally(function() {
// Reenable the GPU preview, if it was active before the modal // Re-enable the GPU preview, if it was active before the modal
if (context.$store.state.autoGpuPreview) { if (context.$store.state.autoGpuPreview) {
context.$root.$emit("globalTogglePreview", true); context.$root.$emit("globalTogglePreview", true);
} }