openflexure-microscope-server/CHANGELOG.md

v3.0.0-beta1 (2025-06-09)

This is the first alpha release of an almost total overhaul of the server, and the underlying LabThings framework. Some highlights are:

• Moved to underlying architecture based on labthings-fastapi. This allows each component of the microscope to be a "Thing", which can interact with each other.
• Current "Things" are camera, stage, autofocus, smart_scan background_detect, auto_recentre_stage, capture, settings_manager, system_control and test
• Overhaul of scanning, removing the predetermined, rectangular area in favour of exploring the sample indefinitely by appending neighbouring xy sites after each capture
• Background detect Thing optionally uses the colour distribution of a user-set "background" image to determine if the field of view is sample or background, and plan future scanning accordingly
• Stitching from openflexure-stitching now part of the scanning protocol, displaying a live updating preview of the current scan, and optionally producing a high res stitched JPEG, DZI and/or TIFF at the end of the scan
• A DZI viewer in the scan tab, allowing stitched images to be opened from within the GUI
• Webapp updates - scan tab added showing thumbnail preview + image count + scan times, gallery removed, tour removed, power tab added,
• Updated to numpy 2
• Automatically delete empty scans

V3 was started in a branch (MR !159)

The following merge requests have been merged into v3:

• !162 Scanning and recentring
• !163 Web app compatibility with v3 API
• !164 Modal progress dialog for TaskSubmitter
• !166 Pseudo spiral scan path and fixed dx-dy inversion
• !167 Restored old log path
• !168 Refactor task submitter
• !169 Run openflexure-stitch on the server
• !170 custom scan name from gui
• !171 Make it possible to manually save all settings to disk
• !172 Updates based on feedback from Rwanda trip
• !173 Autofocus efficiency
• !174 Parallel acquisition and moves
• !175 Improvements to autofocus and stitching in smart_scan.
• !176 Replace tasksubmitter with action-button
• !181 Add configurability to the server
• !191 Migrate to new Blob type and update imports
• !192 Formal Protocol for camera and stage interfaces
• !194 Add more unit tests and fix CI
• !226 Split up ruff format and lint
• !225 Rename exposure (with limit)
• !232 Purge all windows line endings and add check to formatter.
• !231 Reorder tabs
• !211 Use the UV channels only to determine if an area is background or sample
• !228 Stop uvicorn.run disabling existing loggers
• !227 CSM always returns to start on error
• !236 Inverse colours
• !239 Scanning cleanup
• !240 Refactor scanning
• !241 Add some extra rules to ruff, also add a more complete ruff config that fails
• !242 Further scan refactor
• !245 Add a scan path planner that prioritises shorter moves
• !247 Looping autofocus at the start of the scan
• !249 Remove focus_change_acceptable test
• !244 Remove correlation only stitching, now covered by preview stitch
• !250 Thumbnails and accurate durations in scan-list tab
• !253 Regex to count captured images
• !238 Disable tour
• !246 z stacking Thing
• !248 Cleanup zipping
• !255 Remove autofocus test
• !256 New comments on scanning
• !259 Update scan data JSON at end of scan
• !260 Scan tab updates
• !258 Use NumPy v2 and new wheel for opencv
• !230 Lazier regex for finding logs with square brackets
• !263 Clean up of serve_static_files
• !265 Update serve static files
• !267 Update theme.less to prevent automatic changed colour of uk-alert-success
• !268 GUI clean up - button cases, features tab and redo tour
• !269 Action buttons raise an Error from the last message in the log for the Actio
• !257 Added purge_empty_scans() method
• !270 Import openflexure-stitching from pip
• !272 Settling time as global, settle before taking first image in stack
• !261 CSM info in webapp tab
• !275 GUI shutdown clean up
• !276 Ui fixes
• !277 Fix error in purge_empty_scans if the images dir hadn't been created
• !266 Gui dzi viewer
• !278 update labthings-picamera to most recent release
• !279 Simulation fixes
• !281 Change from building with node 15 to node 18
• !282 Defocus and realign simulation axes
• !280 Custom from_index() function in webapp for backwards compatibility before javascrpt at()
• !284 Update labthings-api dep and also the version number for alpha release
• !286 Use same version number for server and webapp, add CI check
• !285 Swapped the order of CSM so y axis is completed first
• !283 Add scan directory to json config
• !287 Update depedencies and changelog prior to v3.0.0-alpha1 release

v2.11.0 (2022-08-08)

New features

• Background detection can now be used in scans, if you have the background-detect extension. (!153)
• Support for Sangaboard firmware v1, via updated sangaboard dependency (!154)
• Fixed links in the OpenAPI documentation describing actions etc. (!155)

v2.10.1 (2022-01-17)

Bug fixes

• Fixed a typo in the serialisation of numpy arrays (#243) (!148)
• Made it possible again to track sharpness while the stage is not moving. This changed when the extra thread was removed from JPEGSharpnessMonitor. (!149)

Minor improvements

• Added a function to measure settling time after a Z move (!149)
• Bumped some INFO logging statements to DEBUG to de-clutter the logs. (!151)

v2.10.0 (2021-08-11)

Documentation improvements

• Many improvements to the generated OpenAPI documentation, resulting in it now validating. (!133)
• Added a user guide and fixed readthedocs. Linked to the API documentation, and store it on the build server with releases. (!140)

Developer changes

• The web app is now in a top-level directory (!122
• OpenAPI descriptions are now easier to download from a merge request. (!139)

Minor bug fixes and improvements

• Removed vestigial error handling code that is now dealt with by LabThings. (!137)
• taskSubmitter buttons in the web app now handle errors more robustly. This fixes a bug where the progress bar would spin indefinitely. (!138)
• Smart stack controls now only appear if the plugin is installed. (!142)
• Javascript dependencies are updated for security. (!141)

v2.10.0b2 (2021-08-11)

New features

• Added support for ImJoy plugins (!120)
• Added a setting to disable the gallery (!120)
• Scan parameters are now remembered in the "capture" pane (!132)
• The Capture pane now contains controls for the smart stack plugin, if installed ((!134))

Developer changes

• CI now fails if Pipfile.lock is out of date (!131)
• The Thing Description and OpenAPI documentation are now valid, and much improved (!133)
• Pytest XML reports are now recorded in the CI pipeline (!135)
• The application is now packaged for every commit on master (!136)

v2.10.0b0 (2021-05-26)

Minor bug fixes and improvements

• Fixed some installation issues with updated Python dependencies (!129)

v2.10.0b0 (2021-05-18)

New features

• Improved auto gain, white balance, and lens shading table correction (!119). Fixes #191. The old auto calibration code has been replaced by a slower, more manual method that is more reliable. Individual parts of the calibration can be run independently (auto gain/exposure, auto white balance, lens shading). This should also improve the experience when using e.g. fluorescence imaging modes.

Developer changes

• Build instructions were updated (!115)
• We've switched from poetry to pipenv to solve dependency management issues on the Raspberry Pi (!124). See the updated repository README for a full breakdown of what configuration information goes where. Closes #218.
• Improved handling of stream settings and origin override (!125, !116, !117) This fixes a longstanding irritation when developing on localhost using npm run serve, where it was necessary to enter the API origin address every time the page was refreshed, and then manually re-enable the web stream. Both these issues are now fixed.
  • The API origin override field in the "dev tools" pane now remembers its value
  • The API origin can be overridden using the URL
  • Stream settings are now remembered in local storage

Minor bug fixes and improvements

• Better error handling for picamerax (!86)
• Add coverage report to mypy (!110)
• MissingStage can now be explicitly selected in microscope configuration (!112)
• The first-run "tour" placement is improved and shouldn't block the interface any more (!114)
  • Fix placement of tour messages for icons (afbb716)
  • This closes #199 and helps with #193
• The IHI interface now explicitly states scan style should be raster (!113)
• Absolute moves are now fixed (!126), closing #220, #221, and #222.
• Fixed a typing error with the camera stage mapping matrix and numpy 1.20 (!127).
• Scans now have a minimum dimension of 1 in each axis, which avoids dividing by zero (!123)

v2.9.3 (2020-12-14)

• Fix #194 (0592d31), closes #194

v2.9.2 (2020-12-09)

• Added more Advanced bitrate controls (d2488c5)
• Watch for broken frames using JPEG end bytes, and log error (6838038)

v2.9.1 (2020-12-07)

• Server: Added mjpeg bitrate to settings (3e2f876)
• JS Client: Replaced MJPEG quality setting with MJPEG bitrate (42e0dfd)

v2.9.0 (2020-12-07)

Developer changes

Extension loader

• Updated default extensions to subclass structure (8d0759e)
  • Updated documentation to subclass-based extensions (bacc111)
  • Added type information to LABTHINGS_EXTENSIONS (b8354d3)
  • Clarified LABTHINGS_EXTENSIONS in docs (f849afd)

Testing

• Added basic unit tests of non-integrated functions (5137d1b)

Static analysis

• Added extra type hints (7ba3f44)
• Added numpy types (63b633b)
• Added type hints to CSM extension (311366c)
• Fixed types in move_rel method (ac667c3)
• Static type analysis (7866ec0)
• Stricter runtime type checks (f2a2d88)

Documentation

• Added better developer notes (a3b1b8a)
• Added changelog generator (de6dbe4)

CI/CD

• Added eslint and cache to CI (f5012cd)
• Added job explanation comments (c1e17de)
• Added poetry to script environment (3814e7d)
• Allow code quality jobs to retry (f15a5c7)
• Fixed JS artifact path (3a90a42)

API functionality

• Added API route to convert LST to PNG (4d40e81)

JS Client

New functionality

• Added log level filter (4d1d0a1)
• Added options to invert navigation steps (d49b34e)
• Added backlash and settle time settings (c0fcd22)
• Added stream and capture quality settings (0f7cecb)

Fixes

• Fixed global state and scrolltotop (ef9f257)
• Fixed individual captures creating 'undefined' dataset (213dec3)
• Fixed onError propagation (08f6532)
• Fixed onScanError (1de6b2a)
• globalUpdateCaptures after scan (599c315)
• Handle missing this.$refs.textboxKey ref (9d04842)
• Fixed IHI tab icon duplication bug (e17366d)

Design/style

• Added custom h4 formatting (4ba4403)
• Rearranged element layout (455b868)
• Rearranged settings and added LST download (f0a3127)

Internal cleanup

• Common watch format (3e783c5)
• Removed unneeded plugin:vue/essential (358d441)
• Removed unused webcomponent support (f187a3a)
• • Moved backlash settings (a2f8158)

Server

Features

• Added API route to convert LST to PNG (4d40e81)
  • Return LST as a file (aba3eb3)

Fixes

• Fixed "classic" autofocus 'PiCameraStreamer' object has no attribute 'annotate_text' error (58b2967)
• Fixed broken dataset rendering if key exists but is empty (fd42e2e)
• Fixed camera settings read (25d3b15)
• Fixed get_locations method reference (d288484)
• Fixed SangaDeltaStage type annotations (9659c45)
• Fixed tile method reference (18a0eed)
• Handle missing endpoints (bc39277)

Internal API changes

• Deprecated camera start_worker and get_frame methods
  • Added start_worker and get_frame aliases for compatibility (bc9b80d)
• Replace top-level actions View with builtin LabThings (421a2e3)
• Updated to LabThings 1.2.1 (249e301)

Internal cleanup

• Remove separate camera stream thread (021745d)
• Cleaned up code layout (da62126)
• Cleaned up main app setup (e464086)
• Cleaned up store and removed unused FoV setting (a1ae947)
• Code formatting (dd81640)
• Fixed unused imports (b2192b2)
• Integrated CSMExtension (9203545)
• Moved frames_iterator scope (3058c67)
• Moved gen() into streams.py (c9c29a7)
• Only use CompositeLock for microscope lock (e433a89)
• Reduced logging level of some mock camera notices (8eff136)
• Removed old console logs (2e34722)
• Removed old picamera_lst_path attribute (85f77fa)
• Removed pointless abstract method implementations (6d1f019)
• Removed unused pynpm package (6819ded)
• Tweaked deprecation warnings (a844efc)
• Updated dependencies (3c3ecd7)

2.8.0 (2020-11-16)

• 2.8.0 (729f101)
• Added documentation comments (3eb2aa8)
• Added frame iterator explanation comment (90ccd0b)
• Added use_video_port argument back to array() (f51d4ff)
• Move stream frame management into FrameStream class (f765540)

2.8.0-beta.3b (2020-11-13)

• 2.8.0-beta.3 (73eda10)
• Add pre-commit (5119e2c)
• Added full TileScanArgs schema (3451b8a)
• Added note on manually running pre-commit (36e837d)
• Added note on pre-commit (0647a03)
• Added rescue to Poetry scripts (1f0b90c)
• Cleaned up Capture schemas (84dcf4f)
• Data-driven tab layout (c645804)
• Extra comments (c48c6d8)
• Formatting (6644819)
• Improved args schema for captures (f681800)
• Moved piexif into captures submodule (7c450b6)
• Moved PyLint config back to pyproject.toml (527bfeb)
• Removed default JSON files and submoduled JSON encoder class (4a9d1c5)
• Removed duplicated code (004ba0b)
• Removed now-redundant TODO comments (c0f9f34)
• Removed old JSONResponse class (b350f62)
• Removed pre-commit from Poetry dependencies (00d30d8)
• Removed PyLint from pre-commit (cb4afba)
• Removed unused files from root (e25c23c)
• Reverted logging style (6fb61e1)
• Style and linting (f4b123c)
• Updated all log strings to new format (9f52521)
• Updated to LabThings 1.1.3 (738f527)
• Warn about unused settings when updating (1e273be)

2.8.0-beta.2 (2020-11-11)

• 2.8.0-beta.2 (8c180ca)
• Added submit button to API origin override form (1e8978d)
• Fixed tab switcher background height (3387c27)
• Scroll to top when pagination switches (fa909db)

2.8.0-beta.1 (2020-11-09)

• 2.8.0-beta.1 (64aa565)
• Added capture count warning (9e887c0)
• Added classes for error sources (7043721)
• Added internet, RAM, and storage checkers (5afff59)
• Added newline symbols (121afb4)
• Added picamera import checker (013aebd)
• Added platform info to output (8349b37)
• Added stage tests and tidied up rescue (0989a50)
• Allow running tasks to be resumed (ac382b7)
• Changed default pollInterval to 1s (5a32e8a)
• Changed newline symbols (061618d)
• Changed reload to 60s timeout (b98522a)
• Fixed duplicated polling timers (fcd9fa4)
• Fixed GPU preview requests happening regardless of settings (5feba6a)
• Fixed ignored kwargs (eae3124)
• Formatting and linting (e416563)
• Formatting and linting (cd61ad3)
• Reduced long-term lock usage (4c61f0c)
• Removed in-memory full metadata from CaptureObject (9dab242)
• Removed port scanning warning (5319beb)
• Removed unused import (cbea2f5)
• Removed unused output (471ee5e)
• Style and linting (de135ab)
• Support tuple-split strings as error messages (f12671f)
• Tidied up logging (06fb81e)
• Upped capture count warning to 10000 (0647acd)

2.8.0-beta.0 (2020-11-03)

• 2.8.0-beta.0 (368f86c)
• Fixed reloading timestamp in new format (86732b8)
• Formatting (32cf2de)
• Removed full capture metadata from top-level resource list (b69c903)
• Simplified build_captures_from_exif (6c51da5)
• Updated client to simplify capture data (982154c)

2.7.0 (2020-10-30)

• 2.7.0 (c58827a)
• Add header to put request to ensure string is sent (97ff8d4)
• Added API LogFileView (f1d0ea5)
• Added download attribute to log file downloader (3ba9718)
• Added extra divider (46120ea)
• Added logger tab (21b150b)
• Added pagination to gallery (e1cca30)
• Always use builtin JPEG thumbnails (10d264c)
• Code formatting (ee8a2fc)
• Code formatting (7f53ff2)
• Code formatting (76094b2)
• Code formatting and linting (250fd07)
• Explicit JPEG thumbnail on capture (09849ce)
• Fixed capture link generator when passed a dict (353450d)
• Fixed gallery overflow (e3a45b8)
• Fixed incorrect import (89b1f9d)
• Fixed node_modules exclusion from Black (13b4c84)
• Fixed tab bar scrolling (279cc03)
• Fixed thumbnail data fallback (32ec2b4)
• Fixed thumbnail generation for video_port captures (1f12154)
• Fixed View import (37c5e20)
• Only save configuration if successfully load new stage type (b111a4e)
• Properly set up ESLint (81ca64e)
• Rearranged main components (eff03bf)
• Removed pointless GPU preview log (3cfbb40)
• Separated move_and_measure locks (da490fd)
• Upgraded to LabThings 1.1.2 (654f722)
• Use application/json for put (2f08bd3)

2.6.5 (2020-10-26)

• 2.6.5 (9ff0aa3)
• Close #186 (729e473), closes #186

2.6.4 (2020-10-26)

• 2.6.4 (7a12dfb)
• Added a settle time to all stage moves (58f7686)
• Added a spiral scan option (ad8fd54)
• Changed frame tracker logging level to debug (9b4031f)
• Close #148 (a258b63), closes #148
• Code formatting (601e51d)
• Fast AF use normal stream (574f4a6)
• Reset tracker frames after each mode (459a8dc)