diff --git a/.gitignore b/.gitignore index 98042836..661ad967 100644 --- a/.gitignore +++ b/.gitignore @@ -90,4 +90,7 @@ openflexure_settings/ /tests/utilities/*.png # Files created by simulator -openflexure/ \ No newline at end of file +openflexure/ + +# js version file +/webapp/src/version.js diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 7dab41ab..45c455b5 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -44,7 +44,7 @@ meta-build-image: - web .javascript: - image: node:15 + image: node:18 before_script: - cd webapp - npm install @@ -58,6 +58,19 @@ meta-build-image: - tags - web +# Check webapp and server have matching version string +version-check: + stage: analysis + #use a base python image as nothing needs to be installed + image: python:3.11 + script: + - ./check_version_sync.py + only: + - branches + - merge_requests + - tags + - web + # Python static analysis with ruff ruff-lint: stage: analysis @@ -145,6 +158,7 @@ build: stage: build extends: .javascript script: + - export NODE_OPTIONS=--openssl-legacy-provider # Build JS application for production - npm run build artifacts: diff --git a/.gitlab/issue_templates/Bug.md b/.gitlab/issue_templates/Bug.md index 86300792..b7e40f0d 100644 --- a/.gitlab/issue_templates/Bug.md +++ b/.gitlab/issue_templates/Bug.md @@ -2,31 +2,35 @@ ## Summary -(Summarize the bug encountered concisely) + ## Configuration + -I'm using: - -**Camera:** (E.g. Raspberry Pi camera v2) - -**Motor controller:** (E.g. Sangaboard, or Arduino Nano) +* **Sever version**: v2 or v3 +* **Release/Branch**: e.g. v3.0.0-alpha1 +* **Hardware Configuration:** + * Raspberry Pi Camera v2 + * Sangabouard v0.5 ## Steps to reproduce -(How one can reproduce the issue - this is very important) + ## Relevant logs and/or screenshots -To access your microscope log, either: + + ## Additional details -(Anything else you think might be relevant to mention) + -/label ~bug +/label ~bug diff --git a/.gitlab/issue_templates/Feature request.md b/.gitlab/issue_templates/Feature request.md index b3230205..5b5f999f 100644 --- a/.gitlab/issue_templates/Feature request.md +++ b/.gitlab/issue_templates/Feature request.md @@ -2,18 +2,18 @@ ## Summary -(Summarize the bug encountered concisely) + -## Configuration - -I'm using: - -**Camera:** (E.g. Raspberry Pi camera v2) - -**Motor controller:** (E.g. Sangaboard, or Arduino Nano) ## Additional details -(Anything else you think might be relevant to mention) + + + +/label ~feature diff --git a/CHANGELOG.md b/CHANGELOG.md index b87bff32..9bfbc0cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,90 @@ +# [v3.0.0-alpha1](https://gitlab.com/openflexure/openflexure-microscope-server/compare/v2.11.0...v3.0.0-alpha1) (2025-06-09) + +This is the first alpha release of an almost total overhaul of the server, and the underlying LabThings framework. Some highlights are: + +* Moved to underlying architecture based on [labthings-fastapi](https://github.com/labthings/labthings-fastapi). This allows each component of the microscope to be a "Thing", which can interact with each other. +* Current "Things" are camera, stage, autofocus, smart_scan background_detect, auto_recentre_stage, capture, settings_manager, system_control and test +* Overhaul of scanning, removing the predetermined, rectangular area in favour of exploring the sample indefinitely by appending neighbouring xy sites after each capture +* Background detect Thing optionally uses the colour distribution of a user-set "background" image to determine if the field of view is sample or background, and plan future scanning accordingly +* Stitching from openflexure-stitching now part of the scanning protocol, displaying a live updating preview of the current scan, and optionally producing a high res stitched JPEG, DZI and/or TIFF at the end of the scan +* A DZI viewer in the scan tab, allowing stitched images to be opened from within the GUI +* Webapp updates - scan tab added showing thumbnail preview + image count + scan times, gallery removed, tour removed, power tab added, +* Updated to numpy 2 +* Automatically delete empty scans + +V3 was started in a branch (MR !159) + +The following merge requests have been merged into v3: + +* !162 Scanning and recentring +* !163 Web app compatibility with v3 API +* !164 Modal progress dialog for TaskSubmitter +* !166 Pseudo spiral scan path and fixed dx-dy inversion +* !167 Restored old log path +* !168 Refactor task submitter +* !169 Run `openflexure-stitch` on the server +* !170 custom scan name from gui +* !171 Make it possible to manually save all settings to disk +* !172 Updates based on feedback from Rwanda trip +* !173 Autofocus efficiency +* !174 Parallel acquisition and moves +* !175 Improvements to autofocus and stitching in `smart_scan`. +* !176 Replace tasksubmitter with action-button +* !181 Add configurability to the server +* !191 Migrate to new Blob type and update imports +* !192 Formal Protocol for camera and stage interfaces +* !194 Add more unit tests and fix CI +* !226 Split up ruff format and lint +* !225 Rename exposure (with limit) +* !232 Purge all windows line endings and add check to formatter. +* !231 Reorder tabs +* !211 Use the UV channels only to determine if an area is background or sample +* !228 Stop uvicorn.run disabling existing loggers +* !227 CSM always returns to start on error +* !236 Inverse colours +* !239 Scanning cleanup +* !240 Refactor scanning +* !241 Add some extra rules to ruff, also add a more complete ruff config that fails +* !242 Further scan refactor +* !245 Add a scan path planner that prioritises shorter moves +* !247 Looping autofocus at the start of the scan +* !249 Remove focus_change_acceptable test +* !244 Remove correlation only stitching, now covered by preview stitch +* !250 Thumbnails and accurate durations in scan-list tab +* !253 Regex to count captured images +* !238 Disable tour +* !246 z stacking Thing +* !248 Cleanup zipping +* !255 Remove autofocus test +* !256 New comments on scanning +* !259 Update scan data JSON at end of scan +* !260 Scan tab updates +* !258 Use NumPy v2 and new wheel for opencv +* !230 Lazier regex for finding logs with square brackets +* !263 Clean up of serve_static_files +* !265 Update serve static files +* !267 Update theme.less to prevent automatic changed colour of uk-alert-success +* !268 GUI clean up - button cases, features tab and redo tour +* !269 Action buttons raise an Error from the last message in the log for the Actio +* !257 Added purge_empty_scans() method +* !270 Import openflexure-stitching from pip +* !272 Settling time as global, settle before taking first image in stack +* !261 CSM info in webapp tab +* !275 GUI shutdown clean up +* !276 Ui fixes +* !277 Fix error in purge_empty_scans if the `images` dir hadn't been created +* !266 Gui dzi viewer +* !278 update labthings-picamera to most recent release +* !279 Simulation fixes +* !281 Change from building with node 15 to node 18 +* !282 Defocus and realign simulation axes +* !280 Custom `from_index()` function in webapp for backwards compatibility before javascrpt `at()` +* !284 Update labthings-api dep and also the version number for alpha release +* !286 Use same version number for server and webapp, add CI check +* !285 Swapped the order of CSM so y axis is completed first +* !283 Add scan directory to json config +* !287 Update depedencies and changelog prior to v3.0.0-alpha1 release + # [v2.11.0](https://gitlab.com/openflexure/openflexure-microscope-server/compare/v2.10.1...v2.11.0) (2022-08-08) ## New features * Background detection can now be used in scans, if you have the background-detect extension. ([!153](https://gitlab.com/openflexure/openflexure-microscope-server/-/merge_requests/153)) diff --git a/README.md b/README.md index 766c4690..f88df89c 100644 --- a/README.md +++ b/README.md @@ -97,19 +97,11 @@ sudo -u openflexure-ws PATH="/var/openflexure/application/openflexure-microscope ### Set up the Javascript environment and build -* The Flask web application, written in Python, serves a web application written in `Vue.js`. This is distributed as part of the built version of the server, hosted on our [build server](https://build.openflexure.org/openflexure-microscope-server/). -* You could extract the pre-built web app from this tarball, which saves you having to set up Node.js. However, it also means you're not able to change the interface, and it's possible your interface will get out of sync with your server. -* Building the web interface will require a valid Node.js installation. If you don't have Node.js (including `npm`) the [Node.js website](https://nodejs.org/en/) offers downloads for all platforms, though see the instructions below for Raspberry Pi. - * To install Node.js on a Raspberry Pi: - * `curl -sL https://deb.nodesource.com/setup_14.x | sudo bash -` - * `sudo apt install nodejs` -* To build the web application (this produces a set of static files, that are served by the Flask webserver) - * `cd webapp` - * `npm install` - * `npm run build` -* To create a Node.js development server (this will help various development tools to display more information, and auto-rebuilds when you change the source files) - * `npm run serve` - * You access the development server on a different port (it's printed on the command line when you run the above command). This means that when it starts up you will need to tell it where the microscope server is, using the "override API origin" field in the page that pops up. If you are running a test server on your computer, this is most likely `http://localhost:5000/`. + +The Labthings-FastAPI server, written in Python, serves a web application written in `Vue.js` (Vue2). This is distributed with the SD card image for the microscope. + +For more details on the web app see the ReadMe in the [webapp](./webapp) directory. + ## Formatting, linting, and tests All of the commands below assume that you are running in the OFM virtual environment, i.e. you have run `ofm activate` on an OpenFlexure SD card, or `source .venv/bin/activate` on Linux, or `.venv/Scripts/activate`on Windows. diff --git a/check_version_sync.py b/check_version_sync.py new file mode 100755 index 00000000..c47365d7 --- /dev/null +++ b/check_version_sync.py @@ -0,0 +1,26 @@ +#! /usr/bin/env python3 + +""" +A script to check that the npm version string matches the python +version string exactly. +""" + +import json +import tomllib +import os + +with open("pyproject.toml", "rb") as toml_f: + pyproject_data = tomllib.load(toml_f) + +with open(os.path.join("webapp", "package.json"), "r", encoding="utf-8") as json_f: + webapp_data = json.load(json_f) + +py_ver = pyproject_data["project"]["version"] +js_ver = webapp_data["version"] + +assert py_ver == js_ver, ( + "The python and javascript version numbers should match.\n" + f"Python version: {py_ver}\n" + f"Javascript version: {js_ver}\n" + "" +) diff --git a/ofm_config_full.json b/ofm_config_full.json index 0e2d9cae..a73e4e40 100644 --- a/ofm_config_full.json +++ b/ofm_config_full.json @@ -7,7 +7,12 @@ "/camera_stage_mapping/": "openflexure_microscope_server.things.camera_stage_mapping:CameraStageMapper", "/system_control/": "openflexure_microscope_server.things.system_control:SystemControlThing", "/settings/": "openflexure_microscope_server.things.settings_manager:SettingsManager", - "/smart_scan/": "openflexure_microscope_server.things.smart_scan:SmartScanThing", + "/smart_scan/": { + "class": "openflexure_microscope_server.things.smart_scan:SmartScanThing", + "kwargs": { + "scans_folder": "/var/openflexure/scans/" + } + }, "/background_detect/": "openflexure_microscope_server.things.background_detect:BackgroundDetectThing", "/capture/": "openflexure_microscope_server.things.capture:CaptureThing" }, diff --git a/ofm_config_simulation.json b/ofm_config_simulation.json index 67721c9b..f982a617 100644 --- a/ofm_config_simulation.json +++ b/ofm_config_simulation.json @@ -7,7 +7,12 @@ "/camera_stage_mapping/": "openflexure_microscope_server.things.camera_stage_mapping:CameraStageMapper", "/system_control/": "openflexure_microscope_server.things.system_control:SystemControlThing", "/settings/": "openflexure_microscope_server.things.settings_manager:SettingsManager", - "/smart_scan/": "openflexure_microscope_server.things.smart_scan:SmartScanThing", + "/smart_scan/": { + "class": "openflexure_microscope_server.things.smart_scan:SmartScanThing", + "kwargs": { + "scans_folder": "./openflexure/scans/" + } + }, "/background_detect/": "openflexure_microscope_server.things.background_detect:BackgroundDetectThing", "/capture/": "openflexure_microscope_server.things.capture:CaptureThing" }, diff --git a/ofm_config_stub.json b/ofm_config_stub.json index 99b51f8e..85aab42e 100644 --- a/ofm_config_stub.json +++ b/ofm_config_stub.json @@ -7,7 +7,12 @@ "/camera_stage_mapping/": "openflexure_microscope_server.things.camera_stage_mapping:CameraStageMapper", "/system_control/": "openflexure_microscope_server.things.system_control:SystemControlThing", "/settings/": "openflexure_microscope_server.things.settings_manager:SettingsManager", - "/smart_scan/": "openflexure_microscope_server.things.smart_scan:SmartScanThing", + "/smart_scan/": { + "class": "openflexure_microscope_server.things.smart_scan:SmartScanThing", + "kwargs": { + "scans_folder": "./openflexure/scans/" + } + }, "/background_detect/": "openflexure_microscope_server.things.background_detect:BackgroundDetectThing", "/capture/": "openflexure_microscope_server.things.capture:CaptureThing" }, diff --git a/pyproject.toml b/pyproject.toml index bb56049b..bfa7b2c5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "openflexure-microscope-server" -version = "3.0.0.dev0" +version = "3.0.0-alpha1" authors = [ { name="Richard Bowman", email="richard.bowman@cantab.net" }, { name="Joel Collins" }, @@ -15,7 +15,7 @@ classifiers = [ "Operating System :: OS Independent", ] dependencies = [ - "labthings-fastapi[server] == 0.0.7", + "labthings-fastapi[server] == 0.0.9", "labthings-sangaboard", "camera-stage-mapping ~= 0.1.10", "opencv-python ~= 4.11.0", @@ -102,22 +102,21 @@ select = [ # "PL", # Pylint (a subset of, catches far less than pylint does) # "B", # Flake8 bugbear "A", # Flake8 builtins checker -# "C", # Flake8 comprehensions + "C", # Flake8 comprehensions # "FIX", #TODO/FIXME's in code. These should be added during develoment for ongoing tasks. # If they are not fixed before merge they should be converted to GitLab issues # "LOG", # Flake8 logging issues (pedantic logger formatting issues can be added with "G") # "T20", # Warns for print statments, production code should log # "PT", # pytest linting -# "RET", # Consistent clear return statments + "RET", # Consistent clear return statements "RSE", # Raise parentheses # "SIM", # Simplifications detected -# "ARG", # unused arguments -# "C90", # McCabe complexity! + "ARG", # unused arguments + "C90", # McCabe complexity! "NPY", # Numpy linting "N", # PEP8 naming -# "DOC", # Enforce argument, return, and raise to be mentioned in docstrings there is - # also "D" but this even warns about docstring punctuation. Perhaps a step too - # far? +# "D", # Docstring checks these may need to be added gradually + "ERA001", # Commented out code! ] ignore = [ diff --git a/ruff-increased-checking.toml b/ruff-increased-checking.toml index 118cfbff..5973655d 100644 --- a/ruff-increased-checking.toml +++ b/ruff-increased-checking.toml @@ -24,16 +24,15 @@ select = [ "LOG", # Flake8 logging issues (pedantic logger formatting issues can be added with "G") "T20", # Warns for print statments, production code should log "PT", # pytest linting - "RET", # Consistent clear return statments + "RET", # Consistent clear return statements "RSE", # Raise parentheses "SIM", # Simplifications detected "ARG", # unused arguments "C90", # McCabe complexity! "NPY", # Numpy linting "N", # PEP8 naming - "DOC", # Enforce argument, return, and raise to be mentioned in docstrings there is - # also "D" but this even warns about docstring punctuation. Perhaps a step too - # far? + "D", # Docstring checks these may need to be added gradually + "ERA001", # Commented out code! ] # Line length is set to 88 for ruff. This is what the formatter aims for diff --git a/src/openflexure_microscope_server/server/__init__.py b/src/openflexure_microscope_server/server/__init__.py index bcf997a3..001b53bc 100644 --- a/src/openflexure_microscope_server/server/__init__.py +++ b/src/openflexure_microscope_server/server/__init__.py @@ -10,17 +10,33 @@ from .legacy_api import add_v2_endpoints from ..logging import configure_logging, retrieve_log, retrieve_log_from_file -def customise_server(server: ThingServer, log_folder: str): +def customise_server(server: ThingServer, log_folder: str, scans_folder: Optional[str]): """Customise the server with additional endpoints, etc.""" configure_logging(log_folder) add_v2_endpoints(server) - add_static_files(server.app) + add_static_files(server.app, scans_folder) # Add an endpoint to get the logs - (directly calling the FastAPI decorator) server.app.get("/log/")(retrieve_log) server.app.get("/logfile/")(retrieve_log_from_file) +def _get_scans_dir(config: dict) -> Optional[str]: + """ + Read the config and return the scans directory. + + Return is None if there is no /smart_scan/ thing loaded. + """ + + if "/smart_scan/" in config["things"]: + try: + return config["things"]["/smart_scan/"]["kwargs"]["scans_folder"] + except KeyError as e: + msg = "Configuration error smart scan should have scans_folder kwarg set" + raise RuntimeError(msg) from e + return None + + def serve_from_cli(argv: Optional[list[str]] = None): """Start the server from the command line""" args = cli.parse_args(argv) @@ -29,11 +45,16 @@ def serve_from_cli(argv: Optional[list[str]] = None): log_config["loggers"]["uvicorn"]["propagate"] = True log_config["loggers"]["uvicorn.access"]["propagate"] = True + # Create server and config vars before trying to configure so they are defined + # if fallback is needed before they are set. + config = None + server = None try: config = cli.config_from_args(args) log_folder = config.get("log_folder", "./openflexure/logs") + scans_folder = _get_scans_dir(config) server = cli.server_from_config(config) - customise_server(server, log_folder) + customise_server(server, log_folder, scans_folder) uvicorn.run( server.app, host=args.host, diff --git a/src/openflexure_microscope_server/server/serve_static_files.py b/src/openflexure_microscope_server/server/serve_static_files.py index 4f5ad96c..123c6d3e 100644 --- a/src/openflexure_microscope_server/server/serve_static_files.py +++ b/src/openflexure_microscope_server/server/serve_static_files.py @@ -1,4 +1,5 @@ import os +from typing import Optional from fastapi.responses import FileResponse, RedirectResponse from fastapi.staticfiles import StaticFiles @@ -22,7 +23,7 @@ def add_static_file(app: FastAPI, fname: str, folder: str) -> None: ) -def add_static_files(app: FastAPI) -> None: +def add_static_files(app: FastAPI, scans_folder: Optional[str]) -> None: """Add the static files responsible for the webapp app to the FastAPI app app: The FastAPI app to add to, in this case the OpenFlexure server @@ -45,9 +46,13 @@ def add_static_files(app: FastAPI) -> None: name=f"static_{fname}", ) - # Mount the scan directory to .../scans/, to allow dzi viewing - app.mount( - "/scans/", - StaticFiles(directory="/var/openflexure/scans/"), - name="scans", - ) + # If scans folder is None, there is not smart scan thing. So nothing to mount. + if scans_folder is not None: + # Mount the scan directory to .../scans/, to allow dzi viewing + if not os.path.isdir(scans_folder): + os.makedirs(scans_folder) + app.mount( + "/scans/", + StaticFiles(directory=scans_folder), + name="scans", + ) diff --git a/src/openflexure_microscope_server/things/auto_recentre_stage.py b/src/openflexure_microscope_server/things/auto_recentre_stage.py index 910a44cf..bbdd5635 100644 --- a/src/openflexure_microscope_server/things/auto_recentre_stage.py +++ b/src/openflexure_microscope_server/things/auto_recentre_stage.py @@ -118,20 +118,12 @@ class RecentringThing(Thing): logging.info( f"Breaking because the highest point is at {np.argmax(sorted_all_heights)} in the list" ) - # plt.plot(sorted_lateral, sorted_all_heights,'.') - # plt.plot(sorted_lateral, quad_fit_func(sorted_lateral)) - # plt.show() + break - else: - if turning_loc < np.min(sorted_lateral): - moves = -1 - elif turning_loc > np.max(sorted_lateral): - moves = 1 - else: - # plt.plot(sorted_lateral, sorted_all_heights,'.') - # plt.plot(sorted_lateral, quad_fit_func(sorted_lateral)) - # plt.show() - pass + if turning_loc < np.min(sorted_lateral): + moves = -1 + elif turning_loc > np.max(sorted_lateral): + moves = 1 # Centre value is replaced by the maximum value recorded in that axis centre[direction] = focused_pos[direction][np.argmax(all_heights)][ diff --git a/src/openflexure_microscope_server/things/autofocus.py b/src/openflexure_microscope_server/things/autofocus.py index 1a14f074..dd4a388a 100644 --- a/src/openflexure_microscope_server/things/autofocus.py +++ b/src/openflexure_microscope_server/things/autofocus.py @@ -105,11 +105,12 @@ class JPEGSharpnessMonitor: stop: int = int(np.argmax(jpeg_times > stage_times[1])) except ValueError as e: if np.sum(jpeg_times > stage_times[0]) == 0: - raise ValueError( - "No images were captured during the move of the stage. Perhaps the camera is not streaming images?" - ) from e - else: - raise e + errmsg = ( + "No images were captured during the move of the stage. " + "Perhaps the camera is not streaming images?" + ) + raise ValueError(errmsg) from e + raise e if stop < 1: stop = len(jpeg_times) logging.debug("changing stop to %s", (stop)) diff --git a/src/openflexure_microscope_server/things/background_detect.py b/src/openflexure_microscope_server/things/background_detect.py index 744ed402..897234c3 100644 --- a/src/openflexure_microscope_server/things/background_detect.py +++ b/src/openflexure_microscope_server/things/background_detect.py @@ -23,8 +23,7 @@ class BackgroundDetectThing(Thing): bd = self.thing_settings.get("background_distributions", None) if bd: return ChannelDistributions(**bd) - else: - return None + return None @background_distributions.setter def background_distributions(self, value: Optional[ChannelDistributions]) -> None: diff --git a/src/openflexure_microscope_server/things/camera/opencv.py b/src/openflexure_microscope_server/things/camera/opencv.py index 3b313485..abb50252 100644 --- a/src/openflexure_microscope_server/things/camera/opencv.py +++ b/src/openflexure_microscope_server/things/camera/opencv.py @@ -83,6 +83,7 @@ class OpenCVCamera(BaseCamera): It's likely to be highly inefficient - raw and/or uncompressed captures using binary image formats will be added in due course. """ + logging.warning(f"OpenCV camera doen't respect {resolution} setting") ret, frame = self.cap.read() if not ret: raise RuntimeError( @@ -100,6 +101,7 @@ class OpenCVCamera(BaseCamera): This function will produce a JPEG image. """ + logging.warning(f"OpenCV camera doen't respect {resolution} setting") frame = self.capture_array() jpeg = cv2.imencode(".jpg", frame)[1].tobytes() exif_dict = { diff --git a/src/openflexure_microscope_server/things/camera/simulation.py b/src/openflexure_microscope_server/things/camera/simulation.py index 5446fd3f..ad0ad1c2 100644 --- a/src/openflexure_microscope_server/things/camera/simulation.py +++ b/src/openflexure_microscope_server/things/camera/simulation.py @@ -104,15 +104,15 @@ class SimulatedCamera(BaseCamera): """Generate an image with blobs based on supplied coordinates""" canvas_width, canvas_height, _ = self.canvas_shape image_width, image_height, _ = self.shape - pos = [x * RATIO for x in pos] + pos = tuple(x * RATIO for x in pos) top_left = ( int(pos[0]) - image_width // 2 - canvas_width // 2, int(pos[1]) - image_height // 2 - canvas_height // 2, ) - focused_image = self.canvas[ - tuple(slice(top_left[i], top_left[i] + self.shape[i]) for i in range(2)) - + (slice(None),) - ] + x_slice = slice(top_left[0], top_left[0] + self.shape[0]) + y_slice = slice(top_left[1], top_left[1] + self.shape[1]) + z_slice = slice(None) + focused_image = self.canvas[(x_slice, y_slice, z_slice)] image = gaussian_filter( focused_image, sigma=np.abs(pos[2]) / 5, @@ -189,6 +189,7 @@ class SimulatedCamera(BaseCamera): It's likely to be highly inefficient - raw and/or uncompressed captures using binary image formats will be added in due course. """ + logging.warning(f"Simulation camera doen't respect {resolution} setting") return self.generate_frame() @thing_action @@ -201,6 +202,7 @@ class SimulatedCamera(BaseCamera): This function will produce a JPEG image. """ + logging.warning(f"Simulation camera doen't respect {resolution} setting") frame = self.capture_array() jpeg = cv2.imencode(".jpg", frame)[1].tobytes() exif_dict = { diff --git a/src/openflexure_microscope_server/things/camera_stage_mapping.py b/src/openflexure_microscope_server/things/camera_stage_mapping.py index 25540169..4e4f1d53 100644 --- a/src/openflexure_microscope_server/things/camera_stage_mapping.py +++ b/src/openflexure_microscope_server/things/camera_stage_mapping.py @@ -87,7 +87,7 @@ def make_hardware_interface( axes = stage.axis_names def pos2dict(pos: Sequence[float]) -> Mapping[str, float]: - return {k: p for k, p in zip(axes, pos)} + return dict(zip(axes, pos)) def dict2pos(posd: Mapping[str, float]) -> Sequence[float]: return tuple(posd[k] for k in axes if k in posd) @@ -228,10 +228,12 @@ class CameraStageMapper(Thing): This performs two 1d calibrations in x and y, then combines their results. """ - logger.info("Calibrating X axis:") - cal_x: dict = self.calibrate_1d(hw, stage, logger, (1, 0, 0)) + # Calibrate y-axis first as it is more likely to fail. + # The x-y difference is due to the camera aspect ratio, not the stage hardware. logger.info("Calibrating Y axis:") cal_y: dict = self.calibrate_1d(hw, stage, logger, (0, 1, 0)) + logger.info("Calibrating X axis:") + cal_x: dict = self.calibrate_1d(hw, stage, logger, (1, 0, 0)) logger.info("Calibration complete, updating metadata.") # Combine X and Y calibrations to make a 2D calibration diff --git a/src/openflexure_microscope_server/things/smart_scan.py b/src/openflexure_microscope_server/things/smart_scan.py index 7d819bb3..50a8c442 100644 --- a/src/openflexure_microscope_server/things/smart_scan.py +++ b/src/openflexure_microscope_server/things/smart_scan.py @@ -114,7 +114,8 @@ def _scan_running(method): class SmartScanThing(Thing): - def __init__(self): + def __init__(self, scans_folder): + self._scans_folder = scans_folder self._preview_stitch_popen = None self._preview_stitch_popen_lock = threading.Lock() self._scan_lock = threading.Lock() @@ -270,11 +271,8 @@ class SmartScanThing(Thing): @property def base_scan_dir(self) -> str: - """This directory will hold all the scans we do.""" - # TODO: This should be determined using sensible configuration. - # If the working directory is `/var/openflexure` this will result - # in scans being saved at `/var/openflexure/scans/` - return "scans" + """The path of the directory where the scans are saved.""" + return self._scans_folder @thing_property def latest_scan_name(self) -> Optional[str]: diff --git a/src/openflexure_microscope_server/things/stage/__init__.py b/src/openflexure_microscope_server/things/stage/__init__.py index cc7c1bc3..81e30ce6 100644 --- a/src/openflexure_microscope_server/things/stage/__init__.py +++ b/src/openflexure_microscope_server/things/stage/__init__.py @@ -80,7 +80,7 @@ class BaseStage(Thing): position = PropertyDescriptor( Mapping[str, int], - {k: 0 for k in _axis_names}, + dict.fromkeys(_axis_names, 0), description="Current position of the stage", readonly=True, observable=True, diff --git a/src/openflexure_microscope_server/things/stage/dummy.py b/src/openflexure_microscope_server/things/stage/dummy.py index 2608da7d..044a6625 100644 --- a/src/openflexure_microscope_server/things/stage/dummy.py +++ b/src/openflexure_microscope_server/things/stage/dummy.py @@ -91,5 +91,5 @@ class DummyStage(BaseStage): It is intended for use after manually or automatically recentring the stage. """ - self.position = {k: 0 for k in self.axis_names} + self.position = dict.fromkeys(self.axis_names, 0) self.instantaneous_position = self.position diff --git a/tests/utilities/scan_test_helpers.py b/tests/utilities/scan_test_helpers.py index 7ee583ed..62bb7f73 100644 --- a/tests/utilities/scan_test_helpers.py +++ b/tests/utilities/scan_test_helpers.py @@ -189,8 +189,7 @@ def get_expected_result_for_example_smart_spiral( """ pkl_fname = os.path.join(THIS_DIR, f"example_smart_spiral_{sample_name}.pkl") with open(pkl_fname, "rb") as pkl_file_obj: - planner = pickle.load(pkl_file_obj) - return planner + return pickle.load(pkl_file_obj) def load_sample_points(sample_name: str): diff --git a/webapp/README.md b/webapp/README.md index 84fd69c2..8af6af95 100644 --- a/webapp/README.md +++ b/webapp/README.md @@ -18,11 +18,20 @@ * Install Node.js (and npm) * Install dependencies with `npm install` -* Node v18 changes SSL, and so you need `$env:NODE_OPTIONS = "--openssl-legacy-provider"` on Windows or `export NODE_OPTIONS=--openssl-legacy-provider` on Linux/MacOS for compatibility. +* Node v18 changes SSL, and so you need a legacy version for compatibility: + * On Windows: `$env:NODE_OPTIONS = "--openssl-legacy-provider"` + * On Linux/MacOS `export NODE_OPTIONS=--openssl-legacy-provider` * Build the static web app with `npm run build` -* Serve a development version with `npm run serve` -We generally run this on the Raspberry Pi (as that is where the Webapp is hosted). If this isn't suitable - for example, if you can't install Node on your microscope due to version conflicts or no internet connection - you can build it on your computer instead, and then copy over the contents of `..\src\openflexure_microscope_server\static` to your Pi (using scp or another file transfer method). +# Developing + +When developing it is useful to use the development server so Vue changes happen instantly without the need to rebuild. To start the development server run: + + npm run serve + +The development server is accessed on a different port from the microscope API. The port to access the development server is printed on the command line when you run the above command. + +When the development webapp starts it cannot locate the microscope API as this is served by the microscope on port 5000. Instead it will present you with a page giving you the option to "override API origin". If you are running a development server on your computer, you should enter `http://localhost:5000/`. ## VS Code and ESLint @@ -39,4 +48,4 @@ To prevent the editor from interfering with ESLint, add to your project `setting "source.fixAll.eslint": true } } -``` \ No newline at end of file +``` diff --git a/webapp/package-lock.json b/webapp/package-lock.json index 38a12b77..0b71f36a 100644 --- a/webapp/package-lock.json +++ b/webapp/package-lock.json @@ -1,12 +1,12 @@ { "name": "openflexure-microscope-jsclient", - "version": "0.0.0", + "version": "3.0.0-alpha1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "openflexure-microscope-jsclient", - "version": "0.0.0", + "version": "3.0.0-alpha1", "license": "GNU General Public License v3.0 ", "dependencies": { "imjoy-core": "^0.13", @@ -27,6 +27,7 @@ "eslint": "^6.8", "eslint-plugin-prettier": "^3.4", "eslint-plugin-vue": "^6.2", + "genversion": "^3.2.0", "less": "^3.13", "less-loader": "^5.0", "mdns-js": "^1.0", @@ -6867,6 +6868,39 @@ "dev": true, "optional": true }, + "node_modules/filelist": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/filelist/-/filelist-1.0.4.tgz", + "integrity": "sha512-w1cEuf3S+DrLCQL7ET6kz+gmlJdbq9J7yXCSjK/OZCPA+qEN1WyF4ZAf0YYJa4/shHJra2t/d/r8SV4Ji+x+8Q==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "minimatch": "^5.0.1" + } + }, + "node_modules/filelist/node_modules/brace-expansion": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.1.tgz", + "integrity": "sha512-XnAIvQ8eM+kC6aULx6wuQiwVsnzsi9d3WxzV3FpWTGA19F621kwdbsAcFKXgKUHZWsy+mY6iL1sHTxWEFCytDA==", + "dev": true, + "license": "MIT", + "dependencies": { + "balanced-match": "^1.0.0" + } + }, + "node_modules/filelist/node_modules/minimatch": { + "version": "5.1.6", + "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-5.1.6.tgz", + "integrity": "sha512-lKwV/1brpG6mBUFHtb7NUmtABCb2WZZmm2wNiOA5hAb8VdCS4B3dtMWyvcoViccwAW/COERjXLt0zP1zXUN26g==", + "dev": true, + "license": "ISC", + "dependencies": { + "brace-expansion": "^2.0.1" + }, + "engines": { + "node": ">=10" + } + }, "node_modules/filesize": { "version": "3.6.1", "resolved": "https://registry.npmjs.org/filesize/-/filesize-3.6.1.tgz", @@ -6984,6 +7018,16 @@ "url": "https://github.com/avajs/find-cache-dir?sponsor=1" } }, + "node_modules/find-package": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/find-package/-/find-package-1.0.0.tgz", + "integrity": "sha512-yVn71XCCaNgxz58ERTl8nA/8YYtIQDY9mHSrgFBfiFtdNNfY0h183Vh8BRkKxD8x9TUw3ec290uJKhDVxqGZBw==", + "dev": true, + "license": "MIT", + "dependencies": { + "parents": "^1.0.1" + } + }, "node_modules/find-up": { "version": "4.1.0", "resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz", @@ -7245,6 +7289,50 @@ "node": ">=6.9.0" } }, + "node_modules/genversion": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/genversion/-/genversion-3.2.0.tgz", + "integrity": "sha512-OIYSX6XYA8PHecLDCTri30hadSZfAjZ8Iq1+BBDXqLWP4dRLuJNLoNjsSWtTpw97IccK2LDWzkEstxAB8GdN7g==", + "dev": true, + "license": "MIT", + "dependencies": { + "commander": "^7.2.0", + "ejs": "^3.1.9", + "find-package": "^1.0.0" + }, + "bin": { + "genversion": "bin/genversion.js" + }, + "engines": { + "node": ">=10.0.0" + } + }, + "node_modules/genversion/node_modules/commander": { + "version": "7.2.0", + "resolved": "https://registry.npmjs.org/commander/-/commander-7.2.0.tgz", + "integrity": "sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 10" + } + }, + "node_modules/genversion/node_modules/ejs": { + "version": "3.1.10", + "resolved": "https://registry.npmjs.org/ejs/-/ejs-3.1.10.tgz", + "integrity": "sha512-UeJmFfOrAQS8OJWPZ4qtgHyWExa088/MtK5UEyoJGFH67cDEXkZSviOiKRCZ4Xij0zxI3JECgYs3oKx+AizQBA==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "jake": "^10.8.5" + }, + "bin": { + "ejs": "bin/cli.js" + }, + "engines": { + "node": ">=0.10.0" + } + }, "node_modules/get-caller-file": { "version": "2.0.5", "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz", @@ -9047,6 +9135,108 @@ "integrity": "sha512-Yljz7ffyPbrLpLngrMtZ7NduUgVvi6wG9RJ9IUcyCd59YQ911PBJphODUcbOVbqYfxe1wuYf/LJ8PauMRwsM/g==", "dev": true }, + "node_modules/jake": { + "version": "10.9.2", + "resolved": "https://registry.npmjs.org/jake/-/jake-10.9.2.tgz", + "integrity": "sha512-2P4SQ0HrLQ+fw6llpLnOaGAvN2Zu6778SJMrCUwns4fOoG9ayrTiZk3VV8sCPkVZF8ab0zksVpS8FDY5pRCNBA==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "async": "^3.2.3", + "chalk": "^4.0.2", + "filelist": "^1.0.4", + "minimatch": "^3.1.2" + }, + "bin": { + "jake": "bin/cli.js" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/jake/node_modules/ansi-styles": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz", + "integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==", + "dev": true, + "license": "MIT", + "dependencies": { + "color-convert": "^2.0.1" + }, + "engines": { + "node": ">=8" + }, + "funding": { + "url": "https://github.com/chalk/ansi-styles?sponsor=1" + } + }, + "node_modules/jake/node_modules/async": { + "version": "3.2.6", + "resolved": "https://registry.npmjs.org/async/-/async-3.2.6.tgz", + "integrity": "sha512-htCUDlxyyCLMgaM3xXg0C0LW2xqfuQ6p05pCEIsXuyQ+a1koYKTuBMzRNwmybfLgvJDMd0r1LTn4+E0Ti6C2AA==", + "dev": true, + "license": "MIT" + }, + "node_modules/jake/node_modules/chalk": { + "version": "4.1.2", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.2.tgz", + "integrity": "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==", + "dev": true, + "license": "MIT", + "dependencies": { + "ansi-styles": "^4.1.0", + "supports-color": "^7.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/chalk/chalk?sponsor=1" + } + }, + "node_modules/jake/node_modules/color-convert": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz", + "integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "color-name": "~1.1.4" + }, + "engines": { + "node": ">=7.0.0" + } + }, + "node_modules/jake/node_modules/color-name": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz", + "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==", + "dev": true, + "license": "MIT" + }, + "node_modules/jake/node_modules/has-flag": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz", + "integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=8" + } + }, + "node_modules/jake/node_modules/supports-color": { + "version": "7.2.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz", + "integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==", + "dev": true, + "license": "MIT", + "dependencies": { + "has-flag": "^4.0.0" + }, + "engines": { + "node": ">=8" + } + }, "node_modules/javascript-stringify": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/javascript-stringify/-/javascript-stringify-2.1.0.tgz", @@ -10798,6 +10988,16 @@ "node": ">=6" } }, + "node_modules/parents": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/parents/-/parents-1.0.1.tgz", + "integrity": "sha512-mXKF3xkoUt5td2DoxpLmtOmZvko9VfFpwRwkKDHSNvgmpLAeBo18YDhcPbBzJq+QLCHMbGOfzia2cX4U+0v9Mg==", + "dev": true, + "license": "MIT", + "dependencies": { + "path-platform": "~0.11.15" + } + }, "node_modules/parse-asn1": { "version": "5.1.6", "resolved": "https://registry.npmjs.org/parse-asn1/-/parse-asn1-5.1.6.tgz", @@ -10924,6 +11124,16 @@ "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==", "dev": true }, + "node_modules/path-platform": { + "version": "0.11.15", + "resolved": "https://registry.npmjs.org/path-platform/-/path-platform-0.11.15.tgz", + "integrity": "sha512-Y30dB6rab1A/nfEKsZxmr01nUotHX0c/ZiIAsCTatEe1CmS5Pm5He7fZ195bPT7RdquoaL8lLxFCMQi/bS7IJg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 0.8.0" + } + }, "node_modules/path-to-regexp": { "version": "0.1.7", "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz", diff --git a/webapp/package.json b/webapp/package.json index 10550b76..1db348dc 100644 --- a/webapp/package.json +++ b/webapp/package.json @@ -1,14 +1,15 @@ { "name": "openflexure-microscope-jsclient", - "version": "0.0.0", + "version": "3.0.0-alpha1", "private": true, "description": "User client for the OpenFlexure Microscope Server", "author": "OpenFlexure (https://www.openflexure.org)", "scripts": { - "serve": "vue-cli-service serve --mode development", - "serve-without-imjoy": "VUE_APP_ENABLE_IMJOY=false vue-cli-service serve --mode development", - "build": "vue-cli-service build --openssl-legacy-provider --mode production", - "build-without-imjoy": "VUE_APP_ENABLE_IMJOY=false vue-cli-service build --mode production", + "gv": "genversion src/version.js", + "serve": "npm run gv; vue-cli-service serve --mode development", + "serve-without-imjoy": "npm run gv; VUE_APP_ENABLE_IMJOY=false vue-cli-service serve --mode development", + "build": "npm run gv; vue-cli-service build --openssl-legacy-provider --mode production", + "build-without-imjoy": "npm run gv; VUE_APP_ENABLE_IMJOY=false vue-cli-service build --mode production", "lint": "vue-cli-service lint", "lint:fix": "vue-cli-service lint --fix" }, @@ -31,6 +32,7 @@ "eslint": "^6.8", "eslint-plugin-prettier": "^3.4", "eslint-plugin-vue": "^6.2", + "genversion": "^3.2.0", "less": "^3.13", "less-loader": "^5.0", "mdns-js": "^1.0", diff --git a/webapp/src/components/labThingsComponents/actionButton.vue b/webapp/src/components/labThingsComponents/actionButton.vue index 7efbbbc9..1defd4dd 100644 --- a/webapp/src/components/labThingsComponents/actionButton.vue +++ b/webapp/src/components/labThingsComponents/actionButton.vue @@ -171,6 +171,26 @@ export default { }, mounted() { + //Define .from_index() as a custom function, working similar to .at() + //For backwards compatibility, not using in-built .at() until updates to Connect + function from_index(n) { + // ToInteger() abstract op + n = Math.trunc(n) || 0; + // Allow negative indexing from the end + if (n < 0) n += this.length; + // OOB access is guaranteed to return undefined + if (n < 0 || n >= this.length) return undefined; + // Otherwise, this is just normal property access + return this[n]; + } + const TypedArray = Reflect.getPrototypeOf(Int8Array); + for (const C of [Array, String, TypedArray]) { + Object.defineProperty(C.prototype, "from_index", + { value: from_index, + writable: true, + enumerable: false, + configurable: true }); + } // Check for already running tasks if (this.taskStarted != true) { this.checkExistingTasks(); @@ -299,14 +319,14 @@ export default { if (!this.progress) this.progress = 1; // Test whether the log is empty or the most recent message is not from an error // If so, return a default message - if (response.data.log.length == 0 | response.data.log.at(-1).levelname != "ERROR") { + if (response.data.log.length == 0 | response.data.log.from_index(-1).levelname != "ERROR") { var message = "Unexpected error, please check the logs"; } // As LabThings Actions add the message from any raised exception to the log, the // last message in the log is the message from the Exception. //If the Exception was raised with no message, use a default. else { - message = response.data.log.at(-1).message||"Unexpected error, please check the logs"; + message = response.data.log.from_index(-1).message||"Unexpected error, please check the logs"; } // Raise an Error with the chosen message reject(new Error(message)); diff --git a/webapp/src/components/tabContentComponents/aboutComponents/statusPane.vue b/webapp/src/components/tabContentComponents/aboutComponents/statusPane.vue index 2d215f06..47cf4463 100644 --- a/webapp/src/components/tabContentComponents/aboutComponents/statusPane.vue +++ b/webapp/src/components/tabContentComponents/aboutComponents/statusPane.vue @@ -25,7 +25,7 @@
Server Version:
- V3 (pre-alpha) + v{{ app_version() }}
diff --git a/webapp/src/components/tabContentComponents/navigateComponents/paneNavigate.vue b/webapp/src/components/tabContentComponents/navigateComponents/paneNavigate.vue index 9d417594..6a01545f 100644 --- a/webapp/src/components/tabContentComponents/navigateComponents/paneNavigate.vue +++ b/webapp/src/components/tabContentComponents/navigateComponents/paneNavigate.vue @@ -386,6 +386,9 @@ export default { margin-left: 5px; margin-right: 5px; width: 3em; + /* Stop Firefox showing input spinners, other + browsers set with block below */ + -moz-appearance: textfield; } /* Chrome, Safari, Edge, Opera */ .numeric-setting-line-input::-webkit-outer-spin-button, diff --git a/webapp/src/main.js b/webapp/src/main.js index 7844a302..ba55b940 100644 --- a/webapp/src/main.js +++ b/webapp/src/main.js @@ -11,6 +11,7 @@ require("vue-tour/dist/vue-tour.css"); // Import MD icons import "material-symbols/outlined.css"; +import version from './version.js' // UIKit overrides UIkit.mixin( @@ -35,6 +36,9 @@ Vue.config.productionTip = false; Vue.mixin({ methods: { + app_version() { + return version; + }, thingDescription(thing) { return this.$store.getters["wot/thingDescription"](thing); },