diff --git a/.codespellrc b/.codespellrc index 84f6269c..73d461c1 100644 --- a/.codespellrc +++ b/.codespellrc @@ -7,7 +7,10 @@ skip = node_modules, __pycache__, ./build, ./dist, - ./apidocs, + ./apidocs/*.html, + ./apidocs/*.js, + ./apidocs/*.json, + ./apidocs/*.css, ./openflexure, ./htmlcov, ./src/openflexure_microscope_server/static, diff --git a/apidocs/dev/README.md b/apidocs/dev/README.md new file mode 100644 index 00000000..3498be6a --- /dev/null +++ b/apidocs/dev/README.md @@ -0,0 +1,9 @@ +# Developer Guidance + +This site provides information that is useful for developing the OpenFlexure Microscope. + +### Viewing and Modifying Locally + +If you want to view these documents in your machine (very useful when editing them). You will need to serve the `apidocs` directory. Navigate to the `apidocs` directory and run `python -m http.server`. + +All developer guidance documentation is written in markdown and then displayed with Docsify. More information on Docsify can be found at https://docsify.js.org. \ No newline at end of file diff --git a/apidocs/dev/_sidebar.md b/apidocs/dev/_sidebar.md new file mode 100644 index 00000000..df1ea285 --- /dev/null +++ b/apidocs/dev/_sidebar.md @@ -0,0 +1,9 @@ +

Developer Documentation Index

+

Python API

+

HTTP API

+ +

Developer Guidance

+ +- [Home](README) +- [Software Architecture](architecture.md) +- [PiCamera Tuning Files](picamera-tuning-files.md) \ No newline at end of file diff --git a/apidocs/dev/architecture.md b/apidocs/dev/architecture.md new file mode 100644 index 00000000..b1e2c6cd --- /dev/null +++ b/apidocs/dev/architecture.md @@ -0,0 +1,11 @@ +# Software Architecture + +## Overview + +The OpenFlexure Microscope Server is a FastAPI server created using Labthings FastAPI. The user interfaces is a Vue.js web app. And thehe server runs on a customised operating system (Raspbian OpenFlexure). + +Here is a high level diagaram of the overall software architecture: + + + +Each repository link should be clickable. The architecture diagram is best viewed in its own tab. diff --git a/apidocs/dev/assets/ChiefRayAngle.png b/apidocs/dev/assets/ChiefRayAngle.png new file mode 100644 index 00000000..56432318 Binary files /dev/null and b/apidocs/dev/assets/ChiefRayAngle.png differ diff --git a/apidocs/dev/assets/GreenEqualisation.png b/apidocs/dev/assets/GreenEqualisation.png new file mode 100644 index 00000000..a0e32735 Binary files /dev/null and b/apidocs/dev/assets/GreenEqualisation.png differ diff --git a/apidocs/dev/assets/SoftwareDiagram.svg b/apidocs/dev/assets/SoftwareDiagram.svg new file mode 100644 index 00000000..0390fdfa --- /dev/null +++ b/apidocs/dev/assets/SoftwareDiagram.svg @@ -0,0 +1,2299 @@ + + + +Other "Things"...Others things, can be installed defined in any installed python package, and included in the config. As of yet there is no way to define a UIThingsExcept for during start up all functionality is defined either within "Things" or in code called by Things. + +Things communicate with each other via thing_slots.1Uvicorn, +An Async Sever Gateway Interface (ASGI) web serverThingsOther custom utilities/librariessystemd service on Raspberry Pi +All python runs in "ofm" virtual environment2OpenFlexure Connect (Electron app) +[or browser]Web App written in Vue.jsCreated by (and served from) the server. Viewed in browser or OpenFlexure Connect1Python code +1LabThings-FastAPI Clientlabthings-fastapi.client. +ThingClientLabThings with automatically make a "ThingClient" class that communicates over HTTP and provides methods for calling all actions and reading properties of any "Thing" running on the server.User's ComputerLocal terminal$ ssh microscope.localv3 Software ArchitecturesshASGIThe systemd service is run on startup. It is managed by systemctl Things add API endpoints to the FastAPI server for communication. +(see add_to_fastapi) Things generate a Thing Description that describe their functionality and data and the HTTP endpoints.HTTPHTTP1OpenFlexure Microscope +ServerThing Descriptionslabthings-fastapi.server. +ThingServerFastAPIsever.app:FastAPILabThings-FastAPI Server11212 + + openflexure +microscope + + Raspbian OpenFlexureThings, are WC3 Web Of Things compliant Python Objects used to add functionality (such as hardware control) with HTTP API endpointsThe server makes use of a number of other python libraries. Some are standard libraries such as NumPy, Pillow, SciPy, etc. Others are specific to OpenFlexureA customised Raspberry Pi operating system, for the embedded Raspberry Pi in the Microscope3OpenFlexure Stitching5For stitching images togetherPlan to remove this in favour of bringing these methods into the microscope where they can be tested.Camera stage mapping8For analysing images to calibrate stage movementMicat7 +For analysing images for optical system calibration1github.com/labthings/labthings-fastapi1gitlab.com/openflexure/openflexure-microscope-server2gitlab.com/openflexure/openflexure-connect3gitlab.com/openflexure/raspberry-pi-os-image-customiser4gitlab.com/openflexure/openflexure-microscope-cli6gitlab.com/bath_open_instrumentation_group/pysangaboard7 +gitlab.com/openflexure/micat5gitlab.com/openflexure/openflexure-stitching8gitlab.com/openflexure/microscope-extensions/camera-stage-mappingRepository URLsPySangaboard6Motor controlPlan to move these scripts into the OS repo.OpenFlexure Microscope CLI +4$ ofm start$ ofm stop$ ofm status$ ofm activate$ ofm logMost commands are a wrapper around systemctlActivates the ofm python environment diff --git a/apidocs/dev/index.html b/apidocs/dev/index.html new file mode 100644 index 00000000..c1e3e1e5 --- /dev/null +++ b/apidocs/dev/index.html @@ -0,0 +1,43 @@ + + + + + + + + OpenFlexure Microscope Server Developer Guidance + + + + + + + +
+ + + + + + + + + diff --git a/apidocs/dev/ofm.css b/apidocs/dev/ofm.css new file mode 100644 index 00000000..f3a7abae --- /dev/null +++ b/apidocs/dev/ofm.css @@ -0,0 +1,1002 @@ +/* This is a theme modified from the vue.css Docsify Theme*/ +@import url("https://fonts.googleapis.com/css2?family=Open+Sans:wght@400;600&display=swap"); + +@import url("https://fonts.googleapis.com/css?family=Roboto+Mono|Source+Sans+Pro:300,400,600"); + +* { + -webkit-font-smoothing: antialiased; + -webkit-overflow-scrolling: touch; + -webkit-tap-highlight-color: rgba(0, 0, 0, 0); + -webkit-text-size-adjust: none; + -webkit-touch-callout: none; + box-sizing: border-box +} + +body:not(.ready) { + overflow: hidden +} + +body:not(.ready) .app-nav, +body:not(.ready)>nav, +body:not(.ready) [data-cloak] { + display: none +} + +div#app { + font-size: 30px; + font-weight: lighter; + margin: 40vh auto; + text-align: center +} + +div#app:empty:before { + content: "Loading..." +} + +img.emoji { + height: 1.2em +} + +img.emoji, +span.emoji { + vertical-align: middle +} + +span.emoji { + font-family: Apple Color Emoji, Segoe UI Emoji, Segoe UI Symbol, Noto Color Emoji; + font-size: 1.2em +} + +.progress { + background-color: #c5247f; + background-color: var(--theme-color, #c5247f); + height: 2px; + left: 0; + position: fixed; + right: 0; + top: 0; + transition: width .2s, opacity .4s; + width: 0; + z-index: 999999 +} + +.search .search-keyword, +.search a:hover { + color: #c5247f; + color: var(--theme-color, #c5247f) +} + +.search .search-keyword { + font-style: normal; + font-weight: 700 +} + +body, +html { + height: 100% +} + +body { + -moz-osx-font-smoothing: grayscale; + -webkit-font-smoothing: antialiased; + color: #34495e; + font-family: 'Open Sans', sans-serif; + font-size: 15px; + letter-spacing: 0; + margin: 0; + overflow-x: hidden +} + +img { + max-width: 100% +} + +a[disabled] { + cursor: not-allowed; + opacity: .6 +} + +kbd { + border: 1px solid #ccc; + border-radius: 3px; + display: inline-block; + font-size: 12px !important; + line-height: 12px; + margin-bottom: 3px; + padding: 3px 5px; + vertical-align: middle +} + +li input[type=checkbox] { + margin: 0 .2em .25em 0; + vertical-align: middle +} + +.app-nav { + margin: 25px 60px 0 0; + position: absolute; + right: 0; + text-align: right; + z-index: 10 +} + +.app-nav.no-badge { + margin-right: 25px +} + +.app-nav p { + margin: 0 +} + +.app-nav>a { + margin: 0 1rem; + padding: 5px 0 +} + +.app-nav li, +.app-nav ul { + display: inline-block; + list-style: none; + margin: 0 +} + +.app-nav a { + color: inherit; + font-size: 16px; + text-decoration: none; + transition: color .3s +} + +.app-nav a.active, +.app-nav a:hover { + color: #c5247f; + color: var(--theme-color, #c5247f) +} + +.app-nav a.active { + border-bottom: 2px solid #c5247f; + border-bottom: 2px solid var(--theme-color, #c5247f) +} + +.app-nav li { + display: inline-block; + margin: 0 1rem; + padding: 5px 0; + position: relative; + cursor: pointer +} + +.app-nav li ul { + background-color: #fff; + border: 1px solid; + border-color: #ddd #ddd #ccc; + border-radius: 4px; + box-sizing: border-box; + display: none; + max-height: calc(100vh - 61px); + overflow-y: auto; + padding: 10px 0; + position: absolute; + right: -15px; + text-align: left; + top: 100%; + white-space: nowrap +} + +.app-nav li ul li { + display: block; + font-size: 14px; + line-height: 1rem; + margin: 8px 14px; + white-space: nowrap +} + +.app-nav li ul a { + display: block; + font-size: inherit; + margin: 0; + padding: 0 +} + +.app-nav li ul a.active { + border-bottom: 0 +} + +.app-nav li:hover ul { + display: block +} + +main { + display: block; + position: relative; + width: 100vw; + height: 100%; + z-index: 0 +} + +main.hidden { + display: none +} + +.anchor { + display: inline-block; + text-decoration: none; + transition: all .3s +} + +.anchor span { + color: #34495e +} + +.anchor:hover { + text-decoration: underline +} + +.sidebar { + border-right: 1px solid rgba(0, 0, 0, .07); + overflow-y: auto; + padding: 40px 0 0; + position: absolute; + top: 0; + bottom: 0; + left: 0; + transition: transform .25s ease-out; + width: 300px; + z-index: 20 +} + +.sidebar>h1, h2 { + margin: 0 0 0 10px; + font-weight: 400; + color: #c5247f; +} + +.sidebar>h1 { + font-size: 1.5rem; +} + +.sidebar>h1 { + font-size: 1.3rem; +} + +.sidebar>h1 a { + color: inherit; + text-decoration: none +} + +.sidebar>h1 .app-nav { + display: block; + position: static +} + +.sidebar .sidebar-nav { + line-height: 2em; + padding-bottom: 40px +} + + +.sidebar .ext { + color: inherit; + margin: 0 0 0 15px; + text-decoration: none +} + +.sidebar li.collapse .app-sub-sidebar { + display: none +} + +.sidebar ul { + margin: 0 0 0 15px; + padding: 0 +} + +.sidebar li>p { + font-weight: 700; + margin: 0 +} + +.sidebar ul, +.sidebar ul li { + list-style: none +} + +.sidebar ul li a { + border-bottom: none; + display: block +} + +.sidebar ul li ul { + padding-left: 20px +} + +.sidebar::-webkit-scrollbar { + width: 4px +} + +.sidebar::-webkit-scrollbar-thumb { + background: transparent; + border-radius: 4px +} + +.sidebar:hover::-webkit-scrollbar-thumb { + background: hsla(0, 0%, 53.3%, .4) +} + +.sidebar:hover::-webkit-scrollbar-track { + background: hsla(0, 0%, 53.3%, .1) +} + +.sidebar-toggle { + background-color: transparent; + background-color: hsla(0, 0%, 100%, .8); + border: 0; + outline: none; + padding: 10px; + position: absolute; + bottom: 0; + left: 0; + text-align: center; + transition: opacity .3s; + width: 284px; + z-index: 30; + cursor: pointer +} + +.sidebar-toggle:hover .sidebar-toggle-button { + opacity: .4 +} + +.sidebar-toggle span { + background-color: #c5247f; + background-color: var(--theme-color, #c5247f); + display: block; + margin-bottom: 4px; + width: 16px; + height: 2px +} + +body.sticky .sidebar, +body.sticky .sidebar-toggle { + position: fixed +} + +.content { + padding-top: 60px; + position: absolute; + top: 0; + right: 0; + bottom: 0; + left: 300px; + transition: left .25s ease +} + +.markdown-section { + margin: 0 auto; + max-width: 80%; + padding: 30px 15px 40px; + position: relative +} + +.markdown-section>* { + box-sizing: border-box; + font-size: inherit +} + +.markdown-section>:first-child { + margin-top: 0 !important +} + +.markdown-section hr { + border: none; + border-bottom: 1px solid #eee; + margin: 2em 0 +} + +.markdown-section iframe { + border: 1px solid #eee; + width: 1px; + min-width: 100% +} + +.markdown-section table { + border-collapse: collapse; + border-spacing: 0; + display: block; + margin-bottom: 1rem; + overflow: auto; + width: 100% +} + +.markdown-section th { + font-weight: 700 +} + +.markdown-section td, +.markdown-section th { + border: 1px solid #ddd; + padding: 6px 13px +} + +.markdown-section tr { + border-top: 1px solid #ccc +} + +.markdown-section p.tip, +.markdown-section tr:nth-child(2n) { + background-color: #f8f8f8 +} + +.markdown-section p.tip { + border-bottom-right-radius: 2px; + border-left: 4px solid #f66; + border-top-right-radius: 2px; + margin: 2em 0; + padding: 12px 24px 12px 30px; + position: relative +} + +.markdown-section p.tip:before { + background-color: #f66; + border-radius: 100%; + color: #fff; + content: "!"; + font-family: Dosis, Source Sans Pro, Helvetica Neue, Arial, sans-serif; + font-size: 14px; + font-weight: 700; + left: -12px; + line-height: 20px; + position: absolute; + height: 20px; + width: 20px; + text-align: center; + top: 14px +} + +.markdown-section p.tip code { + background-color: #efefef +} + +.markdown-section p.tip em { + color: #34495e +} + +.markdown-section p.warn { + background: rgba(66, 185, 131, .1); + border-radius: 2px; + padding: 1rem +} + +.markdown-section ul.task-list>li { + list-style-type: none +} + +body.close .sidebar { + transform: translateX(-300px) +} + +body.close .sidebar-toggle { + width: auto +} + +body.close .content { + left: 0 +} + +@media print { + + .app-nav, + .sidebar, + .sidebar-toggle { + display: none + } +} + +@media screen and (max-width:768px) { + + .sidebar, + .sidebar-toggle { + position: fixed + } + + .app-nav { + margin-top: 16px + } + + .app-nav li ul { + top: 30px + } + + main { + height: auto; + min-height: 100vh; + overflow-x: hidden + } + + .sidebar { + left: -300px; + transition: transform .25s ease-out + } + + .content { + left: 0; + max-width: 100vw; + position: static; + padding-top: 20px; + transition: transform .25s ease + } + + .app-nav { + transition: transform .25s ease-out + } + + .sidebar-toggle { + background-color: transparent; + width: auto; + padding: 30px 30px 10px 10px + } + + body.close .sidebar { + transform: translateX(300px) + } + + body.close .sidebar-toggle { + background-color: hsla(0, 0%, 100%, .8); + transition: background-color 1s; + width: 284px; + padding: 10px + } + + body.close .content { + transform: translateX(300px) + } + + body.close .app-nav { + display: none + } + +} + +section.cover { + position: relative; + align-items: center; + background-position: 50%; + background-repeat: no-repeat; + background-size: cover; + min-height: 100vh; + width: 100%; + display: none +} + +section.cover.show { + display: flex +} + +section.cover.has-mask .mask { + background-color: #fff; + opacity: .8; + position: absolute; + top: 0; + bottom: 0; + width: 100% +} + +section.cover .cover-main { + flex: 1; + margin: 0 16px; + text-align: center; + position: relative +} + +section.cover a { + color: inherit +} + +section.cover a, +section.cover a:hover { + text-decoration: none +} + +section.cover p { + line-height: 1.5rem; + margin: 1em 0 +} + +section.cover h1 { + color: inherit; + font-size: 2.5rem; + font-weight: 300; + margin: .625rem 0 2.5rem; + position: relative; + text-align: center +} + +section.cover h1 a { + display: block +} + +section.cover h1 small { + bottom: -.4375rem; + font-size: 1rem; + position: absolute +} + +section.cover blockquote { + font-size: 1.5rem; + text-align: center +} + +section.cover ul { + line-height: 1.8; + list-style-type: none; + margin: 1em auto; + max-width: 500px; + padding: 0 +} + +section.cover .cover-main>p:last-child a { + border-radius: 2rem; + border: 1px solid #c5247f; + border-color: var(--theme-color, #c5247f); + box-sizing: border-box; + color: #c5247f; + color: var(--theme-color, #c5247f); + display: inline-block; + font-size: 1.05rem; + letter-spacing: .1rem; + margin: .5rem 1rem; + padding: .75em 2rem; + text-decoration: none; + transition: all .15s ease +} + +section.cover .cover-main>p:last-child a:last-child { + background-color: #c5247f; + background-color: var(--theme-color, #c5247f); + color: #fff +} + +section.cover .cover-main>p:last-child a:last-child:hover { + color: inherit; + opacity: .8 +} + +section.cover .cover-main>p:last-child a:hover { + color: inherit +} + +section.cover blockquote>p>a { + border-bottom: 2px solid #c5247f; + border-bottom: 2px solid var(--theme-color, #c5247f); + transition: color .3s +} + +section.cover blockquote>p>a:hover { + color: #c5247f; + color: var(--theme-color, #c5247f) +} + +.sidebar, +body { + background-color: #fff +} + +.sidebar { + color: #364149 +} + +.sidebar li { + margin: 6px 0 +} + +.sidebar ul li a { + color: #505d6b; + font-size: 14px; + font-weight: 400; + overflow: hidden; + text-decoration: none; + text-overflow: ellipsis; + white-space: nowrap +} + +.sidebar ul li a:hover { + text-decoration: underline +} + +.sidebar ul li ul { + padding: 0 +} + +.sidebar ul li.active>a { + border-right: 2px solid; + color: #c5247f; + color: var(--theme-color, #c5247f); + font-weight: 600 +} + +.app-sub-sidebar li:before { + content: "-"; + padding-right: 4px; + float: left +} + +.markdown-section h1, +.markdown-section h2, +.markdown-section h3, +.markdown-section h4, +.markdown-section strong { + color: #2c3e50; + font-weight: 600 +} + +.markdown-section a { + color: #c5247f; + color: var(--theme-color, #c5247f); + font-weight: 600 +} + +.markdown-section h1 { + font-size: 2rem; + margin: 0 0 1rem +} + +.markdown-section h2 { + font-size: 1.75rem; + margin: 45px 0 .8rem +} + +.markdown-section h3 { + font-size: 1.5rem; + margin: 40px 0 .6rem +} + +.markdown-section h4 { + font-size: 1.25rem +} + +.markdown-section h5 { + font-size: 1rem +} + +.markdown-section h6 { + color: #777; + font-size: 1rem +} + +.markdown-section figure, +.markdown-section p { + margin: 1.2em 0 +} + +.markdown-section ol, +.markdown-section p, +.markdown-section ul { + line-height: 1.6rem; + word-spacing: .05rem +} + +.markdown-section ol, +.markdown-section ul { + padding-left: 1.5rem +} + +.markdown-section blockquote { + border-left: 4px solid #c5247f; + border-left: 4px solid var(--theme-color, #c5247f); + color: #858585; + margin: 2em 0; + padding-left: 20px +} + +.markdown-section blockquote p { + font-weight: 600; + margin-left: 0 +} + +.markdown-section iframe { + margin: 1em 0 +} + +.markdown-section em { + color: #7f8c8d +} + +.markdown-section code, +.markdown-section output:after, +.markdown-section pre { + font-family: Roboto Mono, Monaco, courier, monospace +} + +.markdown-section code, +.markdown-section pre { + background-color: #f8f8f8 +} + +.markdown-section output, +.markdown-section pre { + margin: 1.2em 0; + position: relative +} + +.markdown-section output, +.markdown-section pre>code { + border-radius: 2px; + display: block +} + +.markdown-section output:after, +.markdown-section pre>code { + -moz-osx-font-smoothing: initial; + -webkit-font-smoothing: initial +} + +.markdown-section code { + border-radius: 2px; + /*Approx the other side of the colour wheel from OFM pink.*/ + color: #2fbf73; + margin: 0 2px; + padding: 3px 5px; + white-space: pre-wrap +} + +.markdown-section>:not(h1):not(h2):not(h3):not(h4):not(h5):not(h6) code { + font-size: .8rem +} + +.markdown-section pre { + padding: 0 1.4rem; + line-height: 1.5rem; + overflow: auto; + word-wrap: normal +} + +.markdown-section pre>code { + color: #525252; + font-size: .8rem; + padding: 2.2em 5px; + line-height: inherit; + margin: 0 2px; + max-width: inherit; + overflow: inherit; + white-space: inherit +} + +.markdown-section output { + padding: 1.7rem 1.4rem; + border: 1px dotted #ccc +} + +.markdown-section output>:first-child { + margin-top: 0 +} + +.markdown-section output>:last-child { + margin-bottom: 0 +} + +.markdown-section code:after, +.markdown-section code:before, +.markdown-section output:after, +.markdown-section output:before { + letter-spacing: .05rem +} + +.markdown-section output:after, +.markdown-section pre:after { + color: #ccc; + font-size: .6rem; + font-weight: 600; + height: 15px; + line-height: 15px; + padding: 5px 10px 0; + position: absolute; + right: 0; + text-align: right; + top: 0; + content: attr(data-lang) +} + +.token.cdata, +.token.comment, +.token.doctype, +.token.prolog { + color: #8e908c +} + +.token.namespace { + opacity: .7 +} + +.token.boolean, +.token.number { + color: #c76b29 +} + +.token.punctuation { + color: #525252 +} + +.token.property { + color: #c08b30 +} + +.token.tag { + color: #2973b7 +} + +.token.string { + color: #c5247f; + color: var(--theme-color, #c5247f) +} + +.token.selector { + color: #6679cc +} + +.token.attr-name { + color: #2973b7 +} + +.language-css .token.string, +.style .token.string, +.token.entity, +.token.url { + color: #22a2c9 +} + +.token.attr-value, +.token.control, +.token.directive, +.token.unit { + color: #c5247f; + color: var(--theme-color, #c5247f) +} + +.token.function, +.token.keyword { + color: #e96900 +} + +.token.atrule, +.token.regex, +.token.statement { + color: #22a2c9 +} + +.token.placeholder, +.token.variable { + color: #3d8fd1 +} + +.token.deleted { + text-decoration: line-through +} + +.token.inserted { + border-bottom: 1px dotted #202746; + text-decoration: none +} + +.token.italic { + font-style: italic +} + +.token.bold, +.token.important { + font-weight: 700 +} + +.token.important { + color: #c94922 +} + +.token.entity { + cursor: help +} + +code .token { + -moz-osx-font-smoothing: initial; + -webkit-font-smoothing: initial; + min-height: 1.5rem; + position: relative; + left: auto +} \ No newline at end of file diff --git a/apidocs/dev/picamera-tuning-files.md b/apidocs/dev/picamera-tuning-files.md new file mode 100644 index 00000000..000b0194 --- /dev/null +++ b/apidocs/dev/picamera-tuning-files.md @@ -0,0 +1,188 @@ +# Raspberry Pi Camera Tuning Files + +The OpenFlexure Microscope ships its own custom Raspberry Pi Camera tuning files. This page provides more information on these files. + + +## A list of all the settings in the Raspberry Pi Camera (vc4) tuning files. + +Note that VC4 is the patform used by the Pi for with the broadcom ISP. From Pi 5 onwards Raspberry Pi uses their own ISP (PiSP). These have their own tuning file structure. + +More data is available in this datasheet: https://datasheets.raspberrypi.com/camera/raspberry-pi-camera-guide.pdf + +### version: + +This needs to be 2.0. Version one was a normal dictionary of algorithms, rather than this current structure with the algorithms as a list. + +### target + +Needs to be bcm2835. This is the broadcom system on a chip used by vc4. + +### Algorithms + +#### rpi.black_level - Sensor Black Level + +Contains one key-value pair. i.e.: `black_level: 4096` + +Note that this is not the actual black level of the sensor, but the black level once the image has been scaled up to 16 bit. So for a 10-bit sensor this number would need to be divided by 2^6. + +#### rpi.dpc - Defective Pixel Correction + +This has three strength parameters: + +* 0 - None +* 1 - Normal (default) +* 2 - Strong + +#### rpi.lux - Lux Calculation + +This is used to create the lux value. This lux value is not used in the ISP but is sent to the metadata for the image for downstream control algorithms. The camera may crash if this is removed even though we turn off the adaptive algorithms. + +#### rpi.noise - Noise Profile + +This calculates the noise profile for the current images. + +Values from the default tuning files for imx219 and imx477 are: + +``` +"reference_constant": 0, +"reference_slope": 3.67 +``` + +These can be optimised using the methods in the camera tuning tool shipped with libcamera. Though most of the camera tuning tool methods make assumptions that are invalid for a microscope. As such any tuning would need further consideration. + +#### rpi.geq - Green Equalisation + +##### The problems with green equalisation + +This tried to calculate whether adjacent grren pixels should be equal. This interacts very porly with the chief ray angle compensation when using the camera. The imx219 has a lenslet array: + +![](assets/ChiefRayAngle.png) + +Normally as shown in (a) the light comes in at an angle as we get to the edge of the sensor due to how close the lens is to the camera. The lenslet array is designed for this and so if you are in the centre (b), or at the edge (d) the light is focussed onto the correct pixel. + +For a microscope the light comes in perpendicular to the sensor, this makes no difference in the centre but at the edge of the image (e) there is a lot of cross talk between pixels. + +But it makes sense that this also strongly affects the green imbalance (which is something I didn't know about, but should, and will need to read about in more detail so we can improve tuning). If we consider the case of red light, we expect that the red pixels are the only illuminated pixels. If we consider what happens on the sensor in the case of the microscope, I think the light will be focussing on the positions shown: + +![](assets/GreenEqualisation.png) + +So in the case of the diagonals this is illuminating the green in both the red and the blue row. Where as in the areas we were seeing issues the red light will be strongly illuminating the the greens in one row but not the other. + +The result is that for strongly red on the edges every other geen pixel is illuminated causing an overflow. + +##### Turning off the adaptive green equalisation + +The parameters for geeen equalisation are a `slope` and `offset` + +The "green equalisation" algorithm averages Gr and Gb when they "don't look that different". The threshold below which things get averaged is given by + +`threshold = offset + pixel_value * slope` + +As we always want the green pixels to be averaged to remove the effect above. We can set `offset` the maximum 16-bit value (65535). In this case it will always average green pixels, no matter the pixel value or the value of `slope`. + +#### rpi.sdn - Spatial Denoise + +This algorithm is used on the VC4 (Pi 4) but not on PiSP (Pi 5) which uses `rpi.denoise`. A warning on startup asks us to move it into `rpi.denoise` even though we are on VC4 that does not support `rpi.denoise`. + +#### rpi.awb - Automatic White Balance + +Even though we don't use it, camera can't start without it being minimally populated. + +#### rpi.agc - Automatic Gain Control / Automatic Exposure Control + +For calculating the shutter time of the sensor if AeEnable is on. Only the required fields are populated with minimal data as AeEnable is always off. + +#### rpi.alsc - Automatic Lens Shading Correction + +Auto lens shading correction is calibrated by our microscope. This contains the tables for luninance, and colour shading. It can contain multiple tables depending on estimated colour temperature (`ct`). As the microscope has fixed lighting we only provide one set of tables as we don't want the correction to be adaptive. + +We also set `n_iter: 0`. This stops it adding iterative adaptuion. + +As there is only one table, the colour temperature is ignored. We set the colour temperature to 1234 in the tuning file to make it clear that we have not calibrated it on the microscope. We set the colour temperature to 5000 (our actual illumination colour temperature) once recalibrated. + +The tables are flat 1s for luminance on for both sensors. + +For the IMX477 (PiCamera HQ), the Cr and Cb tables are flat, but Cb is 2.0 not 1.0 for all values. This is so that the initial colour gains are set to (1.0, 2.0) + +For the IMX219 (PiCamera v2), the Cr and Cb tables are symmetric tables taken from the top quadrant of a calibrated microscope. They are used to create a good enough correction that the microscope is initially usable for alignment. + +#### rpi.contrast - Contrast + +This does the gamma correction. If `ce_enable` is 1 then the gamma correction is somewhat adaptive. We set this to zero to always use the specified gamma table. + +The table of gamma values is a 1d list of alternating x,y values in the gamma curve. + +We may want to adjust the gamma curve in future. + +#### rpi.ccm - Colour Correction Matrix + +It is possible tp ship different CCMs to be used based on the estimated colour temperatire. We only ship one CCM as our illumination colour temperature is fixed to 5000K, the colour temperature of the LED board. However, the diffuser might affect this. + +The CCMs are different for PiCamera HQ and PiCamera version 2. The default tuning files for these cameras (`/usr/share/libcamera/ipa/rpi/vc4/imx219.json` or `/usr/share/libcamera/ipa/rpi/vc4/imx477.json`) specify CCMs for a number of temperatures but not for 5000K. As such we linearly interpolate between temperatures around 5000K with the following code: + +``` +import numpy as np + +temps = {4650: + np.array([ + 2.18174, -0.70887, -0.47287, + -0.70196, 2.76426, -1.06231, + -0.25157, -0.71978, 1.97135 + ]), + 5858: + np.array( + [ + 2.32392, -0.88421, -0.43971, + -0.63821, 2.58348, -0.94527, + -0.28541, -0.54112, 1.82653 + ]) + } +target_temp = 5000 +temp_low, temp_high = sorted(temps.keys()) +alpha = (target_temp - temp_low) / (temp_high - temp_low) + +print(alpha) + +ccm_interp = (1 - alpha) * temps[temp_low] + alpha * temps[temp_high] +print(np.round(ccm_interp, 6)) + +temps = {4559: np.array( + [ + 2.15423, -0.98143, -0.17279, + -0.38131, 2.14763, -0.76632, + -0.10069, -0.54383, 1.64452 + ]), + 5881:np.array( + [ + 2.18464, -0.95493, -0.22971, + -0.36826, 2.00298, -0.63471, + -0.15219, -0.38055, 1.53274 + ]) + } +target_temp = 5000 +temp_low, temp_high = sorted(temps.keys()) +alpha = (target_temp - temp_low) / (temp_high - temp_low) + +print(alpha) + +ccm_interp = (1 - alpha) * temps[temp_low] + alpha * temps[temp_high] +print(np.round(ccm_interp, 6)) +``` + +##### Calculating our own CCMs + +The CCM calculation is done using a MacBeth target. Lib camera provides a method for this using the Colour Tuning Tool. We will need to adapt this algorithm as they image the target at once and pick out the areas. We have fixed lighting but do not have colour uniformity. + +In the future, with a colour calibration slide, we can take images of each colour with fixed camera settings to create a CCM using the same calculations as the Camera Tuning Tool. + +#### rpi.sharpen - Sharpening + +This is empty. We do not want to be artificially sharpening images as this can add in artifacts. + +#### rpi.hdr - High dynamic range + +Not relevant for us, we don't use high dynamic range capture. + +#### rpi.sync - Software Camera Synchronisation + +Not relevant for us, we only have 1 camera. diff --git a/apidocs/index.html b/apidocs/index.html index 402f8717..ac26f720 100644 --- a/apidocs/index.html +++ b/apidocs/index.html @@ -42,6 +42,7 @@ a { font-weight: 600; border-radius: 6px; width: fit-content; + min-width: 15rem; transition: 0.15s ease; } @@ -61,6 +62,7 @@ a:hover { HTTP API (Swagger) Python API + Developer Guidance diff --git a/docs/images/ChiefRayAngle.png b/docs/images/ChiefRayAngle.png deleted file mode 100644 index 6fc03089..00000000 --- a/docs/images/ChiefRayAngle.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:36ca9e6e8ee0af71a03dee4a6e2d4405c63d54ea9b123046d39c0187402504da -size 96857 diff --git a/docs/images/GreenEqualisation.png b/docs/images/GreenEqualisation.png deleted file mode 100644 index 71cbcd58..00000000 --- a/docs/images/GreenEqualisation.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:8f65ef3712cb9986cb28f40000ddb283e89fe93671b55ad12d8b6fc057afeac9 -size 111804 diff --git a/src/openflexure_microscope_server/things/camera/tuning_files/vc4/README.md b/src/openflexure_microscope_server/things/camera/tuning_files/vc4/README.md index 7881ab22..56fa1d53 100644 --- a/src/openflexure_microscope_server/things/camera/tuning_files/vc4/README.md +++ b/src/openflexure_microscope_server/things/camera/tuning_files/vc4/README.md @@ -1,183 +1,3 @@ -# A list of all the settings in the Raspberry Pi Camera (vc4) tuning files. +## Tuning files. -Note that VC4 is the patform used by the Pi for with the broadcom ISP. From Pi 5 onwards Raspberry Pi uses their own ISP (PiSP). These have their own tuning file structure. - -More data is available in this datasheet: https://datasheets.raspberrypi.com/camera/raspberry-pi-camera-guide.pdf - -## version: - -This needs to be 2.0, version one was a normal dictionary of algorithms, rather than this current structure with the algorithms as a list. - -## target - -Needs to be bcm2835. This is the broadcom system on a chip used by vc4. - -## Algorithms - -### rpi.black_level - Sensor Black Level - -Contains one key-value pair. i.e.: `black_level: 4096` - -Note that this is not the actual black level of the sensor, but the black level once the image has been scaled up to 16 bit. So for a 10-bit sensor this number would need to be divided by 2^6. - -### rpi.dpc - Defective Pixel Correction - -This has three strength parameters: - -* 0 - None -* 1 - Normal (default) -* 2 - Strong - -### rpi.lux - Lux Calculation - -This is used to create the lux value. This lux value is not used in the ISP but is sent to the metadata for the image for downstream control algorithms. The camera may crash if this is removed even though we turn off the adaptive algorithms. - -### rpi.noise - Noise Profile - -This calculates the noise profile for the current images. - -Values from the default tuning files for imx219 and imx477 are: - -``` -"reference_constant": 0, -"reference_slope": 3.67 -``` - -These can be optimised using the methods in the camera tuning tool shipped with libcamera. Though most of the camera tuning tool methods make assumptions that are invalid for a microscope. As such any tuning would need further consideration. - -### rpi.geq - Green Equalisation - -#### The problems with green equalisation - -This tried to calculate whether adjacent grren pixels should be equal. This interacts very porly with the chief ray angle compensation when using the camera. The imx219 has a lenslet array: - -![](../../../../../../docs/images/ChiefRayAngle.png) - -Normally as shown in (a) the light comes in at an angle as we get to the edge of the sensor due to how close the lens is to the camera. The lenslet array is designed for this and so if you are in the centre (b), or at the edge (d) the light is focussed onto the correct pixel. - -For a microscope the light comes in perpendicular to the sensor, this makes no difference in the centre but at the edge of the image (e) there is a lot of cross talk between pixels. - -But it makes sense that this also strongly affects the green imbalance (which is something I didn't know about, but should, and will need to read about in more detail so we can improve tuning). If we consider the case of red light, we expect that the red pixels are the only illuminated pixels. If we consider what happens on the sensor in the case of the microscope, I think the light will be focussing on the positions shown: - -![](../../../../../../docs/images/GreenEqualisation.png) - -So in the case of the diagonals this is illuminating the green in both the red and the blue row. Where as in the areas we were seeing issues the red light will be strongly illuminating the the greens in one row but not the other. - -The result is that for strongly red on the edges every other geen pixel is illuminated causing an overflow. - -#### Turning off the adaptive green equalisation - -The parameters for geeen equalisation are a `slope` and `offset` - -The "green equalisation" algorithm averages Gr and Gb when they "don't look that different". The threshold below which things get averaged is given by - -`threshold = offset + pixel_value * slope` - -As we always want the green pixels to be averaged to remove the effect above. We can set `offset` the maximum 16-bit value (65535). In this case it will always average green pixels, no matter the pixel value or the value of `slope`. - -### rpi.sdn - Spatial Denoise - -This algorithm is used on the VC4 (Pi 4) but not on PiSP (Pi 5) which uses `rpi.denoise`. A warning on startup asks us to move it into `rpi.denoise` even though we are on VC4 that does not support `rpi.denoise`. - -### rpi.awb - Automatic White Balance - -Even though we don't use it, camera can't start without it being minimally populated. - -### rpi.agc - Automatic Gain Control / Automatic Exposure Control - -For calculating the shutter time of the sensor if AeEnable is on. Only the required fields are populated with minimal data as AeEnable is always off. - -### rpi.alsc - Automatic Lens Shading Correction - -Auto lens shading correction is calibrated by our microscope. This contains the tables for luninance, and colour shading. It can contain multiple tables depending on estimated colour temperature (`ct`). As the microscope has fixed lighting we only provide one set of tables as we don't want the correction to be adaptive. - -We also set `n_iter: 0`. This stops it adding iterative adaptuion. - -As there is only one table, the colour temperature is ignored. We set the colour temperature to 1234 in the tuning file to make it clear that we have not calibrated it on the microscope. We set the colour temperature to 5000 (our actual illumination colour temperature) once recalibrated. - -The tables are flat 1s for luminance on for both sensors. - -For the IMX477 (PiCamera HQ), the Cr and Cb tables are flat, but Cb is 2.0 not 1.0 for all values. This is so that the initial colour gains are set to (1.0, 2.0) - -For the IMX219 (PiCamera v2), the Cr and Cb tables are symmetric tables taken from the top quadrant of a calibrated microscope. They are used to create a good enough correction that the microscope is initially usable for alignment. - -### rpi.contrast - Contrast - -This does the gamma correction. If `ce_enable` is 1 then the gamma correction is somewhat adaptive. We set this to zero to always use the specified gamma table. - -The table of gamma values is a 1d list of alternating x,y values in the gamma curve. - -We may want to adjust the gamma curve in future. - -### rpi.ccm - Colour Correction Matrix - -It is possible tp ship different CCMs to be used based on the estimated colour temperatire. We only ship one CCM as our illumination colour temperature is fixed to 5000K, the colour temperature of the LED board. However, the diffuser might affect this. - -The CCMs are different for PiCamera HQ and PiCamera version 2. The default tuning files for these cameras (`/usr/share/libcamera/ipa/rpi/vc4/imx219.json` or `/usr/share/libcamera/ipa/rpi/vc4/imx477.json`) specify CCMs for a number of temperatures but not for 5000K. As such we linearly interpolate between temperatures around 5000K with the following code: - -``` -import numpy as np - -temps = {4650: - np.array([ - 2.18174, -0.70887, -0.47287, - -0.70196, 2.76426, -1.06231, - -0.25157, -0.71978, 1.97135 - ]), - 5858: - np.array( - [ - 2.32392, -0.88421, -0.43971, - -0.63821, 2.58348, -0.94527, - -0.28541, -0.54112, 1.82653 - ]) - } -target_temp = 5000 -temp_low, temp_high = sorted(temps.keys()) -alpha = (target_temp - temp_low) / (temp_high - temp_low) - -print(alpha) - -ccm_interp = (1 - alpha) * temps[temp_low] + alpha * temps[temp_high] -print(np.round(ccm_interp, 6)) - -temps = {4559: np.array( - [ - 2.15423, -0.98143, -0.17279, - -0.38131, 2.14763, -0.76632, - -0.10069, -0.54383, 1.64452 - ]), - 5881:np.array( - [ - 2.18464, -0.95493, -0.22971, - -0.36826, 2.00298, -0.63471, - -0.15219, -0.38055, 1.53274 - ]) - } -target_temp = 5000 -temp_low, temp_high = sorted(temps.keys()) -alpha = (target_temp - temp_low) / (temp_high - temp_low) - -print(alpha) - -ccm_interp = (1 - alpha) * temps[temp_low] + alpha * temps[temp_high] -print(np.round(ccm_interp, 6)) -``` - -#### Calculating our own CCMs - -The CCM calculation is done using a MacBeth target. Lib camera provides a method for this using the Colour Tuning Tool. We will need to adapt this algorithm as they image the target at once and pick out the areas. We have fixed lighting but do not have colour uniformity. - -In the future, with a colour calibration slide, we can take images of each colour with fixed camera settings to create a CCM using the same calculations as the Camera Tuning Tool. - -### rpi.sharpen - Sharpening - -This is empty. We do not want to be artificially sharpening images as this can add in artifacts. - -### rpi.hdr - High dynamic range - -Not relevant for us, we don't use high dynamic range capture. - -### rpi.sync - Software Camera Synchronisation - -Not relevant for us, we only have 1 camera. +More information on the tuning files can be found in the Developer Guidance. This can be found in the `apidocs/dev` directory, and or hosted on GitLab pages. \ No newline at end of file