rahix/openflexure-microscope-server
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'v3-configurable-server' into 'v3'

Browse source 
Add configurability to the server

See merge request openflexure/openflexure-microscope-server!181
This commit is contained in:
Richard Bowman 2024-11-28 10:45:35 +00:00
commit 6f761b0b81
20 changed files with 747 additions and 108 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -79,6 +79,7 @@ openflexure_microscope/cobertura.xml
# labthings settings
/settings/
/openflexure_settings/
# web app build
/src/openflexure_microscope_server/static/

39
README.md
View file

@ -1,8 +1,9 @@
# OpenFlexure Microscope Software
The "server" is the main component of the OpenFlexure Microscope's software. It is responsible for controlling microscope hardware, data management, and allowing it to be controlled locally and over a network.
This repository now includes the web client, which is served from the root of the Python web server.
This software runs on [LabThings-FastAPI](https://github.com/rwb27/labthings-fastapi/), and so most non-microscope functionality is handled by that library.
This repository includes the graphical interface, which is implemented as a web application served from the root of the Python web server. The simplest way to use it is via OpenFlexure eV, which should find your microscope on the network, and display the interface. The microscope's interface can also be accessed at `http://microscope.local:5000/` in a web browser, assuming the hostname of your microscope is `microscope`.
This software runs on [LabThings-FastAPI](https://github.com/labthings/labthings-fastapi/), which creates an HTTP server using FastAPI (which in turn relies on Starlette and pydantic).
## Getting started
@ -12,25 +13,37 @@ The simplest way to set up a microscope is to download the pre-built Raspberry P
There are instructions on how to [use the microscope](https://openflexure.org/projects/microscope/control) once you have installed the software, either from OpenFlexure Connect, or through a web browser.
The web server starts on port 5000 by default, and the microscope SD image uses the hostname "microscope" so you can usually access the web interface at <http://microscope.local:5000/>.
A user guide and developer documentation can be found on [**ReadTheDocs**](https://openflexure-microscope-software.readthedocs.io/), including some installation notes, a link to the HTTP API reference, and guidance for developing extensions.
A user guide and developer documentation **for v2 of the server** can be found on [**ReadTheDocs**](https://openflexure-microscope-software.readthedocs.io/), including some installation notes, a link to the HTTP API reference, and guidance for developing extensions.
More information is also available in the [handbook](https://gitlab.com/openflexure/microscope-handbook/), and in the "development instructions" below.
## Running directly
The Python package provides a command `ofm-microscope-server` that runs the server. This is what is used by `systemd` to run the service. You will need to provide some command-line arguments, see the output of `--help` for an up to date list. See `/etc/systemd/system/openflexure-microscope-server.service` for the command line used to run the server by default on the Raspberry Pi. In general, you are likely to want to specify a configuration file with `-c`, a host (`--host 0.0.0.0` to serve on all addresses) and a port (`--port 5000`). The `--fallback` option will allow the server to start *even if the hardware specified in the configuration file can't load*. This serves an error page, rather than have the server fail. In the future, it should redirect users to a way to fix their configuration.
## Settings
There are 2 important settings files, described in the [docs](https://openflexure-microscope-software.readthedocs.io/en/master/config.html). The paths given below are for the Raspberry Pi installation on our pre-built SD card, and will change if you run on another system:
* `/var/openflexure/settings/microscope_configuration.json`
* Boot-time microscope configuration. Things like the type of camera connected, the stage board, geometry etc.
* Anything that needs to be loaded once as the server starts, and usually doesn't need to be re-written while the server is running
* This configuration file does not change often, and usually only needs to be updated when you change the physical hardware.
* `/var/openflexure/settings/microscope_settings.json`
* Every other persistent setting. Camera settings, calibration data, default capture settings, stream resolution etc.
* This file changes very regularly, and if you need to reset your settings, it's usually this file that you should remove or reset.
The microscope is initially configured by a LabThings config file. This specifies two important things:
* The Python classes (and initialisation arguments) to use for each `Thing`. This sets the type of camera and stage, and enables/disables additional functionality like scanning and autofocus.
* The location of the settings folder, where each Thing can store its settings.
By default, this configuration file should be read from `/var/openflexure/settings/ofm_config.json` when the microscope is run as a service. If it is run at the command line you should specify the configuration using the `-c` command line flag. This configuration file does not change often, and usually only needs to be updated when you change the physical hardware. It may be that this file should be made read-only, particularly in microscopes deployed for e.g. medical applications.
The settings folder is, by default, `/var/openflexure/settings/` on the SD card, or `./settings/` if run elsewhere. It can be changed in the configuration file. This holds every other persistent setting. Camera settings, calibration data, default capture settings, stream resolution and so on. There is one folder per `Thing`, each with their own file. The settings files can change very regularly, and if you need to reset your settings, it's usually these files that you should remove or reset. If you delete one settings file this should reset the corresponding `Thing`, you don't have to delete the whole folder.
# Developer guidelines
## Developing on a Raspberry Pi
The easiest way to work on the software is to build an OpenFlexure Microscope around a Raspberry Pi, using our custom disk image. This includes a pre-installed copy of this server, and a pre-built copy of the web application, so it's ready to use. You can also develop directly on the Raspberry Pi, and this is the best way to test out changes to the Python code using actual hardware. To do this, use the command-line script `ofm develop`. This will replace the server application at `/var/openflexure/application/openflexure-microscope-server/` with a clone of this git repository. You can manage the server with the `ofm` command, using `ofm start`, `ofm stop`, and `ofm restart` to do the respective actions. `ofm serve` will run a debug server that prints its logs and errors to the console.
The easiest way to work on the software is to build an OpenFlexure Microscope around a Raspberry Pi, using our custom disk image. This includes a pre-installed copy of this server, and a pre-built copy of the web application, so it's ready to use. You can also develop directly on the Raspberry Pi, and this is the best way to test out changes to the Python code using actual hardware.
You can manage the server with the `ofm` command, using `ofm start`, `ofm stop`, and `ofm restart` to do the respective actions. Often, stopping the server and running it manually will make errors easier to spot. To do this, run:
```
ofm stop
ofm activate
cd /var/openflexure
sudo -u openflexure-ws openflexure-microscope-server -c settings/microscope_configuration.json
```
Our favourite way of working with the server on a Pi is to follow the instructions above, then open a VSCode Remote session from another computer. This allows you to use your usual developer environment to write code, but everything runs on the Raspberry Pi with real hardware. Note that `ofm develop` uses an `https://` address for the git repository, so you will probably need to change the "remote" URL to your fork of the repository, or to SSH, before you're able to push changes.
@ -67,7 +80,7 @@ To set up a development version of the software (most likely using emulated came
* `python -m venv .venv`
* `source .venv/bin/activate` (on Linux) or `.venv/Scripts/activate` (on Windows)
* `pip install -e .[dev]` (This will install development dependencies. If you don't need these, there is probably a simpler way to run the server than cloning this repo.)
* Finally, run the server: currently you can do this with `sudo systemctl start openflexure-microscope-server` if it's pre-installed on a Raspberry Pi, or `uvicorn --port 5000 openflexure_microscope_server.server:app` to run locally.
* Finally, run the server: currently you can do this with `sudo systemctl start openflexure-microscope-server` if it's pre-installed on a Raspberry Pi, or `openflexure-microscope-server -c ofm_config_stub.json` to run locally.
### Run the server manually on a Raspberry Pi
```

19
ofm_config_full.json Normal file
View file

@ -0,0 +1,19 @@
{
"things": {
"/camera/": "labthings_picamera2.thing:StreamingPiCamera2",
"/stage/": "labthings_sangaboard:SangaboardThing",
"/auto_recentre_stage/": "openflexure_microscope_server.things.auto_recentre_stage:RecentringThing",
"/autofocus/": "openflexure_microscope_server.things.autofocus:AutofocusThing",
"/camera_stage_mapping/": "openflexure_microscope_server.things.camera_stage_mapping:CameraStageMapper",
"/system_control/": "openflexure_microscope_server.things.system_control:SystemControlThing",
"/settings/": "openflexure_microscope_server.things.settings_manager:SettingsManager",
"/smart_scan/": {
"class": "openflexure_microscope_server.things.smart_scan:SmartScanThing",
"kwargs": {
"path_to_openflexure_stitch": "application/openflexure-stitching/.venv/bin/openflexure-stitch"
}
},
"/background_detect/": "openflexure_microscope_server.things.smart_scan:BackgroundDetectThing"
},
"settings_folder": "/var/openflexure/settings/"
}

20
ofm_config_simulation.json Normal file
View file

@ -0,0 +1,20 @@
{
"things": {
"/camera/": "openflexure_microscope_server.things.camera.simulation:SimulatedCamera",
"/stage/": "openflexure_microscope_server.things.stage.dummy:DummyStage",
"/auto_recentre_stage/": "openflexure_microscope_server.things.auto_recentre_stage:RecentringThing",
"/autofocus/": "openflexure_microscope_server.things.autofocus:AutofocusThing",
"/camera_stage_mapping/": "openflexure_microscope_server.things.camera_stage_mapping:CameraStageMapper",
"/system_control/": "openflexure_microscope_server.things.system_control:SystemControlThing",
"/settings/": "openflexure_microscope_server.things.settings_manager:SettingsManager",
"/smart_scan/": {
"class": "openflexure_microscope_server.things.smart_scan:SmartScanThing",
"kwargs": {
"path_to_openflexure_stitch": "application/openflexure-stitching/.venv/bin/openflexure-stitch"
}
},
"/background_detect/": "openflexure_microscope_server.things.smart_scan:BackgroundDetectThing",
"/api_test/": "openflexure_microscope_server.things.test:APITestThing"
},
"settings_folder": "./openflexure_settings/"
}

20
ofm_config_stub.json Normal file
View file

@ -0,0 +1,20 @@
{
"things": {
"/camera/": "openflexure_microscope_server.things.camera.opencv:OpenCVCamera",
"/stage/": "openflexure_microscope_server.things.stage.dummy:DummyStage",
"/auto_recentre_stage/": "openflexure_microscope_server.things.auto_recentre_stage:RecentringThing",
"/autofocus/": "openflexure_microscope_server.things.autofocus:AutofocusThing",
"/camera_stage_mapping/": "openflexure_microscope_server.things.camera_stage_mapping:CameraStageMapper",
"/system_control/": "openflexure_microscope_server.things.system_control:SystemControlThing",
"/settings/": "openflexure_microscope_server.things.settings_manager:SettingsManager",
"/smart_scan/": {
"class": "openflexure_microscope_server.things.smart_scan:SmartScanThing",
"kwargs": {
"path_to_openflexure_stitch": "application/openflexure-stitching/.venv/bin/openflexure-stitch"
}
},
"/background_detect/": "openflexure_microscope_server.things.smart_scan:BackgroundDetectThing",
"/api_test/": "openflexure_microscope_server.things.test:APITestThing"
},
"settings_folder": "./openflexure_settings/"
}

12
pyproject.toml
View file

@ -15,13 +15,14 @@ classifiers = [
"Operating System :: OS Independent",
]
dependencies = [
"labthings-fastapi[server]",
"labthings-fastapi[server] == 0.0.6",
"labthings-sangaboard",
"labthings-picamera2",
"camera-stage-mapping ~= 0.1.6",
"numpy ~= 1.20",
"piexif",
"scipy ~= 1.6",
"opencv-python ~= 4.7.0",
"pillow ~= 10.4",
"anyio ~= 4.0",
]
@ -29,6 +30,13 @@ dependencies = [
dev = [
"labthings-fastapi[dev]",
]
pi = [
"labthings-picamera2 == 0.0.1-dev1",
]
[project.scripts]
openflexure-microscope-server = "openflexure_microscope_server.server:serve_from_cli"
[project.urls]
"Homepage" = "https://openflexure.org/"

71
src/openflexure_microscope_server/server.py
View file

@ -1,71 +0,0 @@
from __future__ import annotations
import logging
from fastapi import Response
from labthings_fastapi.thing_server import ThingServer
from labthings_sangaboard import SangaboardThing
from labthings_picamera2.thing import StreamingPiCamera2
from socket import gethostname
from .things.autofocus import AutofocusThing
from .things.camera_stage_mapping import CameraStageMapper
from .things.system_control import SystemControlThing
from .things.settings_manager import SettingsManager
from .things.auto_recentre_stage import RecentringThing
from .things.smart_scan import SmartScanThing, BackgroundDetectThing
from .things.stitching import Stitcher
from .things.test import APITestThing
from .serve_static_files import add_static_files
from .logging import configure_logging, retrieve_log
configure_logging()
thing_server = ThingServer()
thing_server.add_thing(StreamingPiCamera2(), "/camera/")
thing_server.add_thing(SangaboardThing(), "/stage/")
thing_server.add_thing(RecentringThing(), "/auto_recentre_stage/")
thing_server.add_thing(AutofocusThing(), "/autofocus/")
thing_server.add_thing(CameraStageMapper(), "/camera_stage_mapping/")
thing_server.add_thing(SystemControlThing(), "/system_control/")
thing_server.add_thing(SettingsManager(), "/settings/")
thing_server.add_thing(SmartScanThing("application/openflexure-stitching/.venv/bin/openflexure-stitch"), "/smart_scan/")
thing_server.add_thing(BackgroundDetectThing(), "/background_detect/")
thing_server.add_thing(APITestThing(), "/api_test/")
try:
add_static_files(thing_server.app)
except RuntimeError:
print("Failed to add static files - you will have to do without them!")
# Add an endpoint to get the log
thing_server.app.get("/log/")(retrieve_log)
app = thing_server.app
# TODO: update openflexure connect to make this unnecessary!!
# The endpoints below fool OpenFlexure Connect into thinking we are a
# v2 microscope, so we show up correctly.
# This is necessary until Connect is rebuilt.
@app.get("/routes")
def routes_stub() -> dict[str, dict]:
"""A stub list of routes, used by OF Connect to identify the microscope"""
fake_routes = [
"/api/v2/",
"/api/v2/streams/snapshot",
"/api/v2/instrument/settings/name"
]
return {url: {"url": url, "methods": ["GET"]} for url in fake_routes}
class JPEGResponse(Response):
media_type = "image/jpeg"
@app.get("/api/v2/streams/snapshot")
@app.head("/api/v2/streams/snapshot")
async def thumbnail() -> JPEGResponse:
"""A low-resolution snapshot, for compatibility with OF connect"""
blob = await thing_server.things["/camera/"].lores_mjpeg_stream.grab_frame()
return JPEGResponse(blob)
@app.get("/api/v2/instrument/settings/name")
def get_hostname() -> str:
"""Get the hostname of the device, for compatibility with OF connect"""
return gethostname()

47
src/openflexure_microscope_server/server/__init__.py Normal file
View file

@ -0,0 +1,47 @@
from __future__ import annotations
from typing import Optional
from labthings_fastapi.server import cli, ThingServer
import uvicorn
from .serve_static_files import add_static_files
from .legacy_api import add_v2_endpoints
from ..logging import configure_logging, retrieve_log
def customise_server(server: ThingServer):
"""Customise the server with additional endpoints, etc."""
configure_logging()
add_v2_endpoints(server)
try:
add_static_files(server.app)
except RuntimeError:
print("Failed to add static files - you will have to do without them!")
# Add an endpoint to get the log
server.app.get("/log/")(retrieve_log)
def serve_from_cli(argv: Optional[list[str]] = None):
"""Start the server from the command line"""
args = cli.parse_args(argv)
try:
config, server = None, None
config = cli.config_from_args(args)
server = cli.server_from_config(config)
customise_server(server)
uvicorn.run(server.app, host=args.host, port=args.port)
except BaseException as e:
if args.fallback:
print(f"Error: {e}")
fallback_server = "labthings_fastapi.server.fallback:app"
print(f"Starting fallback server {fallback_server}.")
app = cli.object_reference_to_object(fallback_server)
app.labthings_config = config
app.labthings_server = server
app.labthings_error = e
uvicorn.run(app, host=args.host, port=args.port)
else:
raise e

35
src/openflexure_microscope_server/server/legacy_api.py Normal file
View file

@ -0,0 +1,35 @@
from . import ThingServer
from fastapi import Response
from socket import gethostname
def add_v2_endpoints(thing_server: ThingServer):
app = thing_server.app
# TODO: update openflexure connect to make this unnecessary!!
# The endpoints below fool OpenFlexure Connect into thinking we are a
# v2 microscope, so we show up correctly.
# This is necessary until Connect is rebuilt.
@app.get("/routes")
def routes_stub() -> dict[str, dict]:
"""A stub list of routes, used by OF Connect to identify the microscope"""
fake_routes = [
"/api/v2/",
"/api/v2/streams/snapshot",
"/api/v2/instrument/settings/name"
]
return {url: {"url": url, "methods": ["GET"]} for url in fake_routes}
class JPEGResponse(Response):
media_type = "image/jpeg"
@app.get("/api/v2/streams/snapshot")
@app.head("/api/v2/streams/snapshot")
async def thumbnail() -> JPEGResponse:
"""A low-resolution snapshot, for compatibility with OF connect"""
blob = await thing_server.things["/camera/"].lores_mjpeg_stream.grab_frame()
return JPEGResponse(blob)
@app.get("/api/v2/instrument/settings/name")
def get_hostname() -> str:
"""Get the hostname of the device, for compatibility with OF connect"""
return gethostname()

15
src/openflexure_microscope_server/serve_static_files.py → src/openflexure_microscope_server/server/serve_static_files.py
View file

@ -2,6 +2,7 @@ from fastapi.responses import FileResponse, RedirectResponse
from fastapi.staticfiles import StaticFiles
from fastapi import FastAPI
import os
import pathlib
def add_static_file(app: FastAPI, fname: str, folder: str):
print(f"Adding route for /{fname}")
@ -16,8 +17,18 @@ def add_static_files(app: FastAPI):
#with importlib.resources.as_file(openflexure_microscope_server) as p:
# static_path = p.join("/static/")
#TODO: don't hard code this!
static_path = "/var/openflexure/application/openflexure-microscope-server/src/openflexure_microscope_server/static"
if not os.path.isdir(static_path):
search_paths = [
"/var/openflexure/application/openflexure-microscope-server/src/openflexure_microscope_server/static",
pathlib.Path().absolute() / "application/openflexure-microscope-server/src/openflexure_microscope_server/static",
]
if __file__:
search_paths.append(pathlib.Path(__file__).parent.parent / "static")
for static_path in search_paths:
if os.path.isdir(static_path):
break # stop once one of the paths exists
else:
# If we get to the else: block, no pat was found.
raise RuntimeError("Can't find static files :(")
@app.get("/", response_class=RedirectResponse)

9
src/openflexure_microscope_server/things/auto_recentre_stage.py
View file

@ -1,17 +1,16 @@
import numpy as np
import logging
import time
from labthings_fastapi.thing import Thing
from labthings_fastapi.dependencies.thing import direct_thing_client_dependency
from labthings_fastapi.decorators import thing_action
from labthings_sangaboard import SangaboardThing
from labthings_picamera2.thing import StreamingPiCamera2
from .stage import Stage
from .camera import Camera
from openflexure_microscope_server.things.autofocus import AutofocusThing
from openflexure_microscope_server.things.camera_stage_mapping import CameraStageMapper
StageDep = direct_thing_client_dependency(SangaboardThing, "/stage/")
CamDep = direct_thing_client_dependency(StreamingPiCamera2, "/camera/")
StageDep = direct_thing_client_dependency(Stage, "/stage/")
CamDep = direct_thing_client_dependency(Camera, "/camera/")
CSMDep = direct_thing_client_dependency(CameraStageMapper, "/camera_stage_mapping/")
AutofocusDep = direct_thing_client_dependency(AutofocusThing, "/autofocus/")

10
src/openflexure_microscope_server/things/autofocus.py
View file

@ -19,14 +19,14 @@ from labthings_fastapi.dependencies.thing import direct_thing_client_dependency
from labthings_fastapi.dependencies.blocking_portal import BlockingPortal
from labthings_fastapi.decorators import thing_action
from labthings_fastapi.types.numpy import NDArray
from labthings_picamera2.thing import StreamingPiCamera2
from labthings_sangaboard import SangaboardThing
from .camera import Camera as CameraThing
from .stage import Stage as StageThing
import numpy as np
from pydantic import BaseModel
Stage = direct_thing_client_dependency(SangaboardThing, "/stage/")
Camera = raw_thing_dependency(StreamingPiCamera2)
WrappedCamera = direct_thing_client_dependency(StreamingPiCamera2, "/camera/")
Stage = direct_thing_client_dependency(StageThing, "/stage/")
Camera = raw_thing_dependency(CameraThing)
WrappedCamera = direct_thing_client_dependency(CameraThing, "/camera/")
### Autofocus utilities

106
src/openflexure_microscope_server/things/camera/__init__.py Normal file
View file

@ -0,0 +1,106 @@
"""OpenFlexure Microscope Camera
This module defines the interface for cameras. Any compatible Thing
should enabe the server to work.
See repository root for licensing information.
"""
from __future__ import annotations
import logging
from typing import Literal
from labthings_fastapi.thing import Thing
from labthings_fastapi.decorators import thing_action, thing_property
from labthings_fastapi.dependencies.metadata import GetThingStates
from labthings_fastapi.dependencies.blocking_portal import BlockingPortal
from labthings_fastapi.outputs.mjpeg_stream import MJPEGStreamDescriptor
from labthings_fastapi.outputs.blob import BlobOutput
from labthings_fastapi.types.numpy import NDArray
class JPEGBlob(BlobOutput):
media_type = "image/jpeg"
class Camera(Thing):
"""A Thing representing a camera"""
def __enter__(self):
raise NotImplementedError("Subclasses must implement __enter__")
def __exit__(self, _exc_type, _exc_value, _traceback):
raise NotImplementedError("Subclasses must implement __exit__")
@thing_property
def stream_active(self) -> bool:
"Whether the MJPEG stream is active"
raise NotImplementedError("Subclasses must implement stream_active")
mjpeg_stream = MJPEGStreamDescriptor()
lores_mjpeg_stream = MJPEGStreamDescriptor()
@thing_action
def snap_image(self) -> NDArray:
"""Acquire one image from the camera.
This action cannot run if the camera is in use by a background thread, for
example if a preview stream is running.
"""
return self.capture_array()
@thing_action
def capture_array(
self,
resolution: Literal["lores", "main", "full"] = "main",
) -> NDArray:
"""Acquire one image from the camera and return as an array
This function will produce a nested list containing an uncompressed RGB image.
It's likely to be highly inefficient - raw and/or uncompressed captures using
binary image formats will be added in due course.
"""
raise NotImplementedError("Subclasses must implement capture_array")
@thing_action
def capture_jpeg(
self,
metadata_getter: GetThingStates,
resolution: Literal["lores", "main", "full"] = "main",
) -> JPEGBlob:
"""Acquire one image from the camera and return as a JPEG blob
This function will produce a JPEG image.
"""
raise NotImplementedError("Subclasses must implement capture_jpeg")
@thing_action
def grab_jpeg(
self,
portal: BlockingPortal,
stream_name: Literal["main", "lores"] = "main",
) -> JPEGBlob:
"""Acquire one image from the preview stream and return as an array
This differs from `capture_jpeg` in that it does not pause the MJPEG
preview stream. Instead, we simply return the next frame from that
stream (either "main" for the preview stream, or "lores" for the low
resolution preview). No metadata is returned.
"""
logging.info(f"StreamingPiCamera2.grab_jpeg(stream_name={stream_name}) starting")
stream = (
self.lores_mjpeg_stream if stream_name == "lores" else self.mjpeg_stream
)
frame = portal.call(stream.grab_frame)
logging.info(f"StreamingPiCamera2.grab_jpeg(stream_name={stream_name}) got frame")
return JPEGBlob.from_bytes(frame)
@thing_action
def grab_jpeg_size(
self,
portal: BlockingPortal,
stream_name: Literal["main", "lores"] = "main",
) -> int:
"""Acquire one image from the preview stream and return its size"""
stream = (
self.lores_mjpeg_stream if stream_name == "lores" else self.mjpeg_stream
)
return portal.call(stream.next_frame_size)

112
src/openflexure_microscope_server/things/camera/opencv.py Normal file
View file

@ -0,0 +1,112 @@
"""OpenFlexure Microscope OpenCV Camera
This module defines a camera Thing that uses OpenCV's
`VideoCapture`.
See repository root for licensing information.
"""
from __future__ import annotations
import io
import json
import logging
from typing import Literal, Optional
from threading import Thread
import cv2
import piexif
from labthings_fastapi.utilities import get_blocking_portal
from labthings_fastapi.decorators import thing_action, thing_property
from labthings_fastapi.dependencies.metadata import GetThingStates
from labthings_fastapi.outputs.mjpeg_stream import MJPEGStreamDescriptor
from labthings_fastapi.outputs.blob import BlobOutput
from labthings_fastapi.types.numpy import NDArray
from . import Camera
class JPEGBlob(BlobOutput):
media_type = "image/jpeg"
class OpenCVCamera(Camera):
"""A Thing representing an OpenCV camera"""
def __init__(self, camera_index: int=0):
self.camera_index = camera_index
self._capture_thread: Optional[Thread] = None
self._capture_enabled = False
def __enter__(self):
self.cap = cv2.VideoCapture(self.camera_index)
self._capture_enabled = True
self._capture_thread = Thread(target=self._capture_frames)
self._capture_thread.start()
return self
def __exit__(self, _exc_type, _exc_value, _traceback):
if self.stream_active:
self._capture_enabled = False
self._capture_thread.join()
self.cap.release()
@thing_property
def stream_active(self) -> bool:
"Whether the MJPEG stream is active"
if self._capture_enabled and self._capture_thread:
return self._capture_thread.is_alive()
return False
mjpeg_stream = MJPEGStreamDescriptor()
lores_mjpeg_stream = MJPEGStreamDescriptor()
def _capture_frames(self):
portal = get_blocking_portal(self)
while self._capture_enabled:
ret, frame = self.cap.read()
if not ret:
logging.error(f"Failed to capture frame from camera {self.camera_index}")
break
jpeg = cv2.imencode(".jpg", frame)[1].tobytes()
self.mjpeg_stream.add_frame(jpeg, portal)
jpeg_lores = cv2.imencode(".jpg", cv2.resize(frame, (320, 240)))[1].tobytes()
self.lores_mjpeg_stream.add_frame(jpeg_lores, portal)
@thing_action
def capture_array(
self,
resolution: Literal["main", "full"] = "full",
) -> NDArray:
"""Acquire one image from the camera and return as an array
This function will produce a nested list containing an uncompressed RGB image.
It's likely to be highly inefficient - raw and/or uncompressed captures using
binary image formats will be added in due course.
"""
ret, frame = self.cap.read()
if not ret:
raise RuntimeError(f"Failed to capture frame from camera {self.camera_index}")
return frame
@thing_action
def capture_jpeg(
self,
metadata_getter: GetThingStates,
resolution: Literal["main", "full"] = "main",
) -> JPEGBlob:
"""Acquire one image from the camera and return as a JPEG blob
This function will produce a JPEG image.
"""
frame = self.capture_array()
jpeg = cv2.imencode(".jpg", frame)[1].tobytes()
exif_dict = {
"Exif": {
piexif.ExifIFD.UserComment: json.dumps(metadata_getter()).encode("utf-8")
},
"GPS": {},
"Interop": {},
"1st": {},
"thumbnail": None,
}
output = io.BytesIO()
piexif.insert(piexif.dump(exif_dict), jpeg, output)
return JPEGBlob.from_bytes(output.getvalue())

176
src/openflexure_microscope_server/things/camera/simulation.py Normal file
View file

@ -0,0 +1,176 @@
"""OpenFlexure Microscope OpenCV Camera
This module defines a Thing that is responsible for using the stage and
camera together to perform an autofocus routine.
See repository root for licensing information.
"""
from __future__ import annotations
import io
import json
import logging
from typing import Literal, Optional
from threading import Thread
import time
import cv2
import numpy as np
import piexif
from labthings_fastapi.utilities import get_blocking_portal
from labthings_fastapi.decorators import thing_action, thing_property
from labthings_fastapi.dependencies.metadata import GetThingStates
from labthings_fastapi.outputs.mjpeg_stream import MJPEGStreamDescriptor
from labthings_fastapi.types.numpy import NDArray
from labthings_fastapi.server import ThingServer
from . import Camera, JPEGBlob
from ..stage import Stage
class SimulatedCamera(Camera):
"""A Thing representing an OpenCV camera"""
shape = (600, 800, 3)
glyph_shape = (51, 51, 3)
canvas_shape = (3000, 4000, 3)
frame_interval = 0.1
_stage: Optional[Stage] = None
_server: Optional[ThingServer] = None
def __init__(self):
self._capture_thread: Optional[Thread] = None
self._capture_enabled = False
self.generate_sprites()
self.generate_blobs()
self.generate_canvas()
def generate_sprites(self):
"""Generate sprites to populate the image"""
self.sprites = []
black = np.zeros(self.glyph_shape, dtype=np.uint8)
x = np.arange(black.shape[0])
y = np.arange(black.shape[1])
rr = np.sqrt((x[:, None] - np.mean(x))**2 + (y[None, :] - np.mean(y))**2)
for i in [5, 7, 9, 11, 13, 15]:
sprite = black.copy()
sprite[rr < i] = 255
self.sprites.append(sprite)
def generate_blobs(self, N: int = 1000):
"""Generate coordinates of blobs
Blobs are characterised by X, Y, sprite
We also generate a KD tree to rapidly find blobs in an image
"""
self.blobs = np.zeros((N, 3))
rng = np.random.default_rng()
w = np.max(self.glyph_shape)
self.blobs[:, 0] = rng.uniform(w/2, self.canvas_shape[0]-w/2, N)
self.blobs[:, 1] = rng.uniform(w/2, self.canvas_shape[1]-w/2, N)
self.blobs[:, 2] = rng.choice(len(self.sprites), N)
def generate_canvas(self):
"""Generate a blank canvas"""
self.canvas = np.zeros(self.canvas_shape, dtype=np.uint8)
self.canvas[...] = 255
w, h, _ = self.glyph_shape
for x, y, sprite in self.blobs:
self.canvas[
int(x) - w//2:int(x) - w//2 + w,
int(y) - h//2:int(y) - h//2 + h,
] -= self.sprites[int(sprite)]
def generate_image(self, pos: tuple[int, int]):
"""Generate an image with blobs based on supplied coordinates"""
cw, ch, _ = self.canvas_shape
w, h, _ = self.shape
tl = (int(pos[0]) - w//2 - cw//2, int(pos[1]) - h//2 - ch//2)
return self.canvas[
tuple(slice(tl[i],tl[i] + self.shape[i]) for i in range(2)) + (slice(None),)
]
def attach_to_server(self, server: ThingServer, path: str):
self._server = server
return super().attach_to_server(server, path)
def get_stage_position(self):
if not self._stage and self._server:
self._stage = self._server.things["/stage/"]
return self._stage.instantaneous_position
def generate_frame(self):
"""Generate a frame with blobs based on the stage coordinates"""
try:
pos = self.get_stage_position()
except Exception as e:
print(f"Failed to get stage position: {e}")
pos = {"x": 0, "y": 0}
return self.generate_image((pos["x"]/10, pos["y"]/10))
def __enter__(self):
self._capture_enabled = True
self._capture_thread = Thread(target=self._capture_frames)
self._capture_thread.start()
return self
def __exit__(self, _exc_type, _exc_value, _traceback):
if self.stream_active:
self._capture_enabled = False
self._capture_thread.join()
@thing_property
def stream_active(self) -> bool:
"Whether the MJPEG stream is active"
if self._capture_enabled and self._capture_thread:
return self._capture_thread.is_alive()
return False
mjpeg_stream = MJPEGStreamDescriptor()
lores_mjpeg_stream = MJPEGStreamDescriptor()
def _capture_frames(self):
portal = get_blocking_portal(self)
while self._capture_enabled:
time.sleep(self.frame_interval)
frame = self.generate_frame()
jpeg = cv2.imencode(".jpg", frame)[1].tobytes()
self.mjpeg_stream.add_frame(jpeg, portal)
jpeg_lores = cv2.imencode(".jpg", cv2.resize(frame, (320, 240)))[1].tobytes()
self.lores_mjpeg_stream.add_frame(jpeg_lores, portal)
@thing_action
def capture_array(
self,
resolution: Literal["main", "full"] = "full",
) -> NDArray:
"""Acquire one image from the camera and return as an array
This function will produce a nested list containing an uncompressed RGB image.
It's likely to be highly inefficient - raw and/or uncompressed captures using
binary image formats will be added in due course.
"""
return self.generate_frame()
@thing_action
def capture_jpeg(
self,
metadata_getter: GetThingStates,
resolution: Literal["main", "full"] = "main",
) -> JPEGBlob:
"""Acquire one image from the camera and return as a JPEG blob
This function will produce a JPEG image.
"""
frame = self.capture_array()
jpeg = cv2.imencode(".jpg", frame)[1].tobytes()
exif_dict = {
"Exif": {
piexif.ExifIFD.UserComment: json.dumps(metadata_getter()).encode("utf-8")
},
"GPS": {},
"Interop": {},
"1st": {},
"thumbnail": None,
}
output = io.BytesIO()
piexif.insert(piexif.dump(exif_dict), jpeg, output)
return JPEGBlob.from_bytes(output.getvalue())

13
src/openflexure_microscope_server/things/camera_stage_mapping.py
View file

@ -22,8 +22,8 @@ from camera_stage_mapping.camera_stage_calibration_1d import (
image_to_stage_displacement_from_1d,
)
from camera_stage_mapping.camera_stage_tracker import Tracker
from labthings_picamera2.thing import StreamingPiCamera2
from labthings_sangaboard import SangaboardThing
from .camera import Camera as CameraThing
from .stage import Stage as StageThing
from labthings_fastapi.dependencies.thing import direct_thing_client_dependency
from labthings_fastapi.dependencies.invocation import InvocationCancelledError, InvocationLogger
@ -31,8 +31,8 @@ from labthings_fastapi.types.numpy import NDArray, denumpify, DenumpifyingDict
from labthings_fastapi.decorators import thing_action, thing_property
from labthings_fastapi.thing import Thing
Camera = direct_thing_client_dependency(StreamingPiCamera2, "/camera/")
Stage = direct_thing_client_dependency(SangaboardThing, "/stage/")
Camera = direct_thing_client_dependency(CameraThing, "/camera/")
Stage = direct_thing_client_dependency(StageThing, "/stage/")
CoordinateType = Tuple[float, float, float]
XYCoordinateType = Tuple[float, float]
@ -87,7 +87,10 @@ def make_hardware_interface(
return downsample(downsample_factor, img)
def settle() -> None:
time.sleep(0.2)
camera.capture_metadata
try:
camera.capture_metadata # This discards frames on a picamera
except AttributeError:
pass # Don't raise an error for other cameras (may consider grabbing a frame)
return HardwareInterfaceModel(
move=move, get_position=get_position, grab_image=grab_image, settle=settle, grab_image_downsampling=downsample_factor
)

2
src/openflexure_microscope_server/things/settings_manager.py
View file

@ -13,7 +13,7 @@ from fastapi import Depends, HTTPException, Request
from labthings_fastapi.dependencies.metadata import GetThingStates
from labthings_fastapi.thing import Thing
from labthings_fastapi.decorators import thing_action, thing_property
from labthings_fastapi.thing_server import find_thing_server, ThingServer
from labthings_fastapi.server import find_thing_server, ThingServer
from labthings_fastapi.dependencies.invocation import InvocationLogger

9
src/openflexure_microscope_server/things/smart_scan.py
View file

@ -28,14 +28,15 @@ from labthings_fastapi.dependencies.thing import direct_thing_client_dependency
from labthings_fastapi.dependencies.invocation import CancelHook, InvocationLogger, InvocationCancelledError
from labthings_fastapi.decorators import thing_action, thing_property, fastapi_endpoint
from labthings_fastapi.outputs.blob import BlobOutput
from labthings_sangaboard import SangaboardThing
from labthings_picamera2.thing import StreamingPiCamera2
from .camera import Camera
from .stage import Stage
from openflexure_microscope_server.things.autofocus import AutofocusThing
from openflexure_microscope_server.things.camera_stage_mapping import CameraStageMapper
from openflexure_microscope_server.things.auto_recentre_stage import RecentringThing
StageDep = direct_thing_client_dependency(SangaboardThing, "/stage/")
CamDep = direct_thing_client_dependency(StreamingPiCamera2, "/camera/")
CamDep = direct_thing_client_dependency(Camera, "/camera/")
StageDep = direct_thing_client_dependency(Stage, "/stage/")
CSMDep = direct_thing_client_dependency(CameraStageMapper, "/camera_stage_mapping/")
AutofocusDep = direct_thing_client_dependency(AutofocusThing, "/autofocus/")
RecentreStage = direct_thing_client_dependency(RecentringThing, "/auto_recentre_stage/")

63
src/openflexure_microscope_server/things/stage/__init__.py Normal file
View file

@ -0,0 +1,63 @@
from __future__ import annotations
from labthings_fastapi.descriptors.property import PropertyDescriptor
from labthings_fastapi.thing import Thing
from labthings_fastapi.decorators import thing_action, thing_property
from labthings_fastapi.dependencies.invocation import CancelHook
from collections.abc import Sequence, Mapping
class Stage(Thing):
"""A dummy stage for testing purposes
This stage should work similarly to a Sangaboard stage, but without any
hardware attached.
"""
_axis_names = ("x", "y", "z")
@thing_property
def axis_names(self) -> Sequence[str]:
"""The names of the stage's axes, in order."""
return self._axis_names
position = PropertyDescriptor(
Mapping[str, int],
{k: 0 for k in _axis_names},
description="Current position of the stage",
readonly=True,
observable=True,
)
moving = PropertyDescriptor(
bool,
False,
description="Whether the stage is in motion",
readonly=True,
observable=True,
)
@property
def thing_state(self):
"""Summary metadata describing the current state of the stage"""
return {
"position": self.position
}
@thing_action
def move_relative(self, cancel: CancelHook, block_cancellation: bool=False, **kwargs: Mapping[str, int]):
"""Make a relative move. Keyword arguments should be axis names."""
raise NotImplementedError("Subclasses should implement this method")
@thing_action
def move_absolute(self, cancel: CancelHook, block_cancellation: bool=False, **kwargs: Mapping[str, int]):
"""Make an absolute move. Keyword arguments should be axis names."""
raise NotImplementedError("Subclasses should implement this method")
@thing_action
def set_zero_position(self):
"""Make the current position zero in all axes
This action does not move the stage, but resets the position to zero.
It is intended for use after manually or automatically recentring the
stage.
"""
raise NotImplementedError("Subclasses should implement this method")

76
src/openflexure_microscope_server/things/stage/dummy.py Normal file
View file

@ -0,0 +1,76 @@
from __future__ import annotations
from labthings_fastapi.decorators import thing_action
from labthings_fastapi.dependencies.invocation import CancelHook, InvocationCancelledError
from collections.abc import Mapping
import time
from . import Stage
class DummyStage(Stage):
"""A dummy stage for testing purposes
This stage should work similarly to a Sangaboard stage, but without any
hardware attached.
"""
def __enter__(self):
self.instantaneous_position = self.position
def __exit__(self, _exc_type, _exc_value, _traceback):
pass
@thing_action
def move_relative(self, cancel: CancelHook, block_cancellation: bool=False, **kwargs: Mapping[str, int]):
"""Make a relative move. Keyword arguments should be axis names."""
displacement = [kwargs.get(k, 0) for k in self.axis_names]
self.moving = True
try:
fraction_complete = 0.0
dt = 0.001
max_displacement = max(abs(v) for v in displacement)
start_time = time.time()
while time.time() - start_time < dt * max_displacement:
if block_cancellation:
time.sleep(dt)
else:
cancel.sleep(dt)
fraction_complete = (time.time() - start_time) / (dt * max_displacement)
self.instantaneous_position = {
k: self.position[k] + int(fraction_complete * v)
for k, v in zip(self.axis_names, displacement)
}
fraction_complete = 1.0
except InvocationCancelledError as e:
# If the move has been cancelled, stop it but don't handle the exception.
# We need the exception to propagate in order to stop any calling tasks,
# and to mark the invocation as "cancelled" rather than stopped.
raise e
finally:
self.moving=False
self.position = {
k: self.position[k] + int(fraction_complete * v)
for k, v in zip(self.axis_names, displacement)
}
self.instantaneous_position = self.position
@thing_action
def move_absolute(self, cancel: CancelHook, block_cancellation: bool=False, **kwargs: Mapping[str, int]):
"""Make an absolute move. Keyword arguments should be axis names."""
displacement = {
k: int(v) - self.position[k]
for k, v in kwargs.items()
if k in self.axis_names
}
self.move_relative(cancel, block_cancellation=block_cancellation, **displacement)
@thing_action
def set_zero_position(self):
"""Make the current position zero in all axes
This action does not move the stage, but resets the position to zero.
It is intended for use after manually or automatically recentring the
stage.
"""
self.position = {k: 0 for k in self.axis_names}
self.instantaneous_position = self.position