stream¶
Models for miniscope_io.stream_daq
- pydantic model miniscope_io.models.stream.StreamBufferHeader¶
Refinements of
BufferHeaderforStreamDaqShow JSON schema
{ "title": "StreamBufferHeader", "description": "Refinements of :class:`.BufferHeader` for\n:class:`~miniscope_io.stream_daq.StreamDaq`", "type": "object", "properties": { "linked_list": { "title": "Linked List", "type": "integer" }, "frame_num": { "title": "Frame Num", "type": "integer" }, "buffer_count": { "title": "Buffer Count", "type": "integer" }, "frame_buffer_count": { "title": "Frame Buffer Count", "type": "integer" }, "write_buffer_count": { "title": "Write Buffer Count", "type": "integer" }, "dropped_buffer_count": { "title": "Dropped Buffer Count", "type": "integer" }, "timestamp": { "title": "Timestamp", "type": "integer" }, "write_timestamp": { "title": "Write Timestamp", "type": "integer" }, "pixel_count": { "title": "Pixel Count", "type": "integer" }, "battery_voltage": { "title": "Battery Voltage", "type": "integer" }, "ewl_pos": { "title": "Ewl Pos", "type": "integer" } }, "required": [ "linked_list", "frame_num", "buffer_count", "frame_buffer_count", "write_buffer_count", "dropped_buffer_count", "timestamp", "write_timestamp", "pixel_count", "battery_voltage", "ewl_pos" ] }
- pydantic model miniscope_io.models.stream.StreamBufferHeaderFormat¶
Refinements of
BufferHeaderFormatforStreamDaq- Parameters:
Show JSON schema
{ "title": "StreamBufferHeaderFormat", "description": "Refinements of :class:`.BufferHeaderFormat` for\n:class:`~miniscope_io.stream_daq.StreamDaq`\n\nParameters\n----------\npixel_count: int\n Number of pixels in the buffer.\nbattery_voltage: int\n Battery voltage. Mapping to mV will be documented in device documentation.\newl_pos: int\n Electrical wetting lens position.", "type": "object", "properties": { "linked_list": { "title": "Linked List", "type": "integer" }, "frame_num": { "title": "Frame Num", "type": "integer" }, "buffer_count": { "title": "Buffer Count", "type": "integer" }, "frame_buffer_count": { "title": "Frame Buffer Count", "type": "integer" }, "write_buffer_count": { "title": "Write Buffer Count", "type": "integer" }, "dropped_buffer_count": { "title": "Dropped Buffer Count", "type": "integer" }, "timestamp": { "title": "Timestamp", "type": "integer" }, "write_timestamp": { "title": "Write Timestamp", "type": "integer" }, "pixel_count": { "title": "Pixel Count", "type": "integer" }, "battery_voltage": { "title": "Battery Voltage", "type": "integer" }, "ewl_pos": { "title": "Ewl Pos", "type": "integer" } }, "required": [ "linked_list", "frame_num", "buffer_count", "frame_buffer_count", "write_buffer_count", "dropped_buffer_count", "timestamp", "write_timestamp", "pixel_count", "battery_voltage", "ewl_pos" ] }
- pydantic model miniscope_io.models.stream.StreamDevConfig¶
Format model used to parse DAQ configuration yaml file (examples are in ./config) The model attributes are key-value pairs needed for reconstructing frames from data streams.
- Parameters:
device (str) – Interface hardware used for receiving data. Current options are “OK” (Opal Kelly XEM 7310) and “UART” (generic UART-USB converters). Only “OK” is supported at the moment.
bitstream (str, optional) – Required when device is “OK”. The configuration bitstream file to upload to the Opal Kelly board. This uploads a Manchester decoder HDL and different bitstream files are required to configure different data rates and bit polarity. This is a binary file synthesized using Vivado, and details for generating this file will be provided in later updates.
port (str, optional) – Required when device is “UART”. COM port connected to the UART-USB converter.
baudrate (Optional[int]) – Required when device is “UART”. Baudrate of the connection to the UART-USB converter.
frame_width (int) – Frame width of transferred image. This is used to reconstruct image.
frame_height (int) – Frame height of transferred image. This is used to reconstruct image.
fs (int) – Framerate of acquired stream
preamble (str) – 32-bit preamble used to locate the start of each buffer. The header and image data follows this preamble. This is used as a hex but imported as a string because yaml doesn’t support hex format.
header_len (int, optional) – Length of header in bits. (For 32-bit words, 32 * number of words) This is useful when not all the variable/words in the header are defined in
MetadataHeaderFormat. The user is responsible to ensure that header_len is larger than the largest bit position defined inMetadataHeaderFormatotherwise unexpected behavior might occur.pix_depth (int, optional) – Bit-depth of each pixel, by default 8.
buffer_block_length (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. Number of blocks per each data buffer. This is required to calculate the number of pixels contained in one data buffer.
block_size (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. Number of 32-bit words per data block. This is required to calculate the number of pixels contained in one data buffer.
num_buffers (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. This is the number of buffers that the source microcontroller cycles around. This isn’t strictly required for data reconstruction but useful for debugging.
reverse_header_bits (bool, optional) – If True, reverse the bits within each byte of the header. Default is False.
reverse_header_bytes (bool, optional) – If True, reverse the byte order within each 32-bit word of the header. This is used for handling endianness in systems where the byte order needs to be swapped. Default is False.
reverse_payload_bits (bool, optional) – If True, reverse the bits within each byte of the payload. Default is False.
reverse_payload_bytes (bool, optional) – If True, reverse the byte order within each 32-bit word of the payload. This is used for handling endianness in systems where the byte order needs to be swapped. Default is False.
..todo:: – Move port (for USART) to a user config area. This should make this pure device config.
Show JSON schema
{ "title": "StreamDevConfig", "description": "Format model used to parse DAQ configuration yaml file (examples are in ./config)\nThe model attributes are key-value pairs needed for reconstructing frames from data streams.\n\nParameters\n----------\ndevice: str\n Interface hardware used for receiving data.\n Current options are \"OK\" (Opal Kelly XEM 7310) and \"UART\" (generic UART-USB converters).\n Only \"OK\" is supported at the moment.\nbitstream: str, optional\n Required when device is \"OK\".\n The configuration bitstream file to upload to the Opal Kelly board.\n This uploads a Manchester decoder HDL and different bitstream files are required\n to configure different data rates and bit polarity.\n This is a binary file synthesized using Vivado,\n and details for generating this file will be provided in later updates.\nport: str, optional\n Required when device is \"UART\".\n COM port connected to the UART-USB converter.\nbaudrate: Optional[int]\n Required when device is \"UART\".\n Baudrate of the connection to the UART-USB converter.\nframe_width: int\n Frame width of transferred image. This is used to reconstruct image.\nframe_height: int\n Frame height of transferred image. This is used to reconstruct image.\nfs: int\n Framerate of acquired stream\npreamble: str\n 32-bit preamble used to locate the start of each buffer.\n The header and image data follows this preamble.\n This is used as a hex but imported as a string because yaml doesn't support hex format.\nheader_len : int, optional\n Length of header in bits. (For 32-bit words, 32 * number of words)\n This is useful when not all the variable/words in the header are defined in\n :class:`.MetadataHeaderFormat`.\n The user is responsible to ensure that `header_len` is larger than the largest bit\n position defined in :class:`.MetadataHeaderFormat`\n otherwise unexpected behavior might occur.\npix_depth : int, optional\n Bit-depth of each pixel, by default 8.\nbuffer_block_length: int\n Defines the data buffer structure. This value needs to match the Miniscope firmware.\n Number of blocks per each data buffer.\n This is required to calculate the number of pixels contained in one data buffer.\nblock_size: int\n Defines the data buffer structure. This value needs to match the Miniscope firmware.\n Number of 32-bit words per data block.\n This is required to calculate the number of pixels contained in one data buffer.\nnum_buffers: int\n Defines the data buffer structure. This value needs to match the Miniscope firmware.\n This is the number of buffers that the source microcontroller cycles around.\n This isn't strictly required for data reconstruction but useful for debugging.\nreverse_header_bits : bool, optional\n If True, reverse the bits within each byte of the header.\n Default is False.\nreverse_header_bytes : bool, optional\n If True, reverse the byte order within each 32-bit word of the header.\n This is used for handling endianness in systems where the byte order needs to be swapped.\n Default is False.\nreverse_payload_bits : bool, optional\n If True, reverse the bits within each byte of the payload.\n Default is False.\nreverse_payload_bytes : bool, optional\n If True, reverse the byte order within each 32-bit word of the payload.\n This is used for handling endianness in systems where the byte order needs to be swapped.\n Default is False.\n\n..todo::\n Move port (for USART) to a user config area. This should make this pure device config.", "type": "object", "properties": { "device": { "enum": [ "OK", "UART" ], "title": "Device", "type": "string" }, "bitstream": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "title": "Bitstream" }, "port": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Port" }, "baudrate": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Baudrate" }, "frame_width": { "title": "Frame Width", "type": "integer" }, "frame_height": { "title": "Frame Height", "type": "integer" }, "fs": { "default": 20, "title": "Fs", "type": "integer" }, "preamble": { "format": "binary", "title": "Preamble", "type": "string" }, "header_len": { "title": "Header Len", "type": "integer" }, "pix_depth": { "default": 8, "title": "Pix Depth", "type": "integer" }, "buffer_block_length": { "title": "Buffer Block Length", "type": "integer" }, "block_size": { "title": "Block Size", "type": "integer" }, "num_buffers": { "title": "Num Buffers", "type": "integer" }, "reverse_header_bits": { "default": false, "title": "Reverse Header Bits", "type": "boolean" }, "reverse_header_bytes": { "default": false, "title": "Reverse Header Bytes", "type": "boolean" }, "reverse_payload_bits": { "default": false, "title": "Reverse Payload Bits", "type": "boolean" }, "reverse_payload_bytes": { "default": false, "title": "Reverse Payload Bytes", "type": "boolean" }, "runtime": { "allOf": [ { "$ref": "#/$defs/StreamDevRuntime" } ], "default": { "serial_buffer_queue_size": 10, "frame_buffer_queue_size": 5, "image_buffer_queue_size": 5, "plot": { "history": 500, "keys": [ "timestamp", "buffer_count", "frame_buffer_count" ], "update_ms": 1000 }, "csvwriter": { "buffer": 100 } } } }, "$defs": { "CSVWriterConfig": { "description": "Configuration for :class:`miniscope_io.io.BufferedCSVWriter`", "properties": { "buffer": { "default": 100, "description": "Buffer length for CSV writer", "title": "Buffer", "type": "integer" } }, "title": "CSVWriterConfig", "type": "object" }, "StreamDevRuntime": { "description": "Runtime configuration for :class:`.StreamDaq`\n\nIncluded within :class:`.StreamDevConfig` to separate config that is not\nunique to the device, but how that device is controlled at runtime.", "properties": { "serial_buffer_queue_size": { "default": 10, "description": "Buffer length for serial data reception in streamDaq", "title": "Serial Buffer Queue Size", "type": "integer" }, "frame_buffer_queue_size": { "default": 5, "description": "Buffer length for storing frames in streamDaq", "title": "Frame Buffer Queue Size", "type": "integer" }, "image_buffer_queue_size": { "default": 5, "description": "Buffer length for storing images in streamDaq", "title": "Image Buffer Queue Size", "type": "integer" }, "plot": { "anyOf": [ { "$ref": "#/$defs/StreamPlotterConfig" }, { "type": "null" } ], "default": { "keys": [ "timestamp", "buffer_count", "frame_buffer_count" ], "update_ms": 1000, "history": 500 }, "description": "Configuration for plotting header data as it is collected. If ``None``, use the default params in StreamPlotter. Note that this does *not* control whether header metadata is plotted during capture, for enabling/disabling, use the ``show_metadata`` kwarg in the capture method" }, "csvwriter": { "anyOf": [ { "$ref": "#/$defs/CSVWriterConfig" }, { "type": "null" } ], "default": { "buffer": 100 }, "description": "Default configuration for writing header data to a CSV file. If ``None``, use the default params in BufferedCSVWriter. Note that this does *not* control whether header metadata is written during capture, for enabling/disabling, use the ``metadata`` kwarg in the capture method." } }, "title": "StreamDevRuntime", "type": "object" }, "StreamPlotterConfig": { "description": "Configuration for :class:`miniscope_io.plots.headers.StreamPlotter`", "properties": { "keys": { "description": "Keys to specify what fields of the given model to plot", "items": { "type": "string" }, "title": "Keys", "type": "array" }, "update_ms": { "default": 1000, "description": "Update rate for stream header plots in milliseconds", "title": "Update Ms", "type": "integer" }, "history": { "default": 500, "description": "Number of stream headers to plot", "title": "History", "type": "integer" } }, "required": [ "keys" ], "title": "StreamPlotterConfig", "type": "object" } }, "required": [ "device", "frame_width", "frame_height", "preamble", "header_len", "buffer_block_length", "block_size", "num_buffers" ] }
- Fields:
- Validators:
- field runtime: StreamDevRuntime = StreamDevRuntime(serial_buffer_queue_size=10, frame_buffer_queue_size=5, image_buffer_queue_size=5, plot=StreamPlotterConfig(keys=['timestamp', 'buffer_count', 'frame_buffer_count'], update_ms=1000, history=500), csvwriter=CSVWriterConfig(buffer=100))¶
- pydantic model miniscope_io.models.stream.StreamDevRuntime¶
Runtime configuration for
StreamDaqIncluded within
StreamDevConfigto separate config that is not unique to the device, but how that device is controlled at runtime.Show JSON schema
{ "title": "StreamDevRuntime", "description": "Runtime configuration for :class:`.StreamDaq`\n\nIncluded within :class:`.StreamDevConfig` to separate config that is not\nunique to the device, but how that device is controlled at runtime.", "type": "object", "properties": { "serial_buffer_queue_size": { "default": 10, "description": "Buffer length for serial data reception in streamDaq", "title": "Serial Buffer Queue Size", "type": "integer" }, "frame_buffer_queue_size": { "default": 5, "description": "Buffer length for storing frames in streamDaq", "title": "Frame Buffer Queue Size", "type": "integer" }, "image_buffer_queue_size": { "default": 5, "description": "Buffer length for storing images in streamDaq", "title": "Image Buffer Queue Size", "type": "integer" }, "plot": { "anyOf": [ { "$ref": "#/$defs/StreamPlotterConfig" }, { "type": "null" } ], "default": { "keys": [ "timestamp", "buffer_count", "frame_buffer_count" ], "update_ms": 1000, "history": 500 }, "description": "Configuration for plotting header data as it is collected. If ``None``, use the default params in StreamPlotter. Note that this does *not* control whether header metadata is plotted during capture, for enabling/disabling, use the ``show_metadata`` kwarg in the capture method" }, "csvwriter": { "anyOf": [ { "$ref": "#/$defs/CSVWriterConfig" }, { "type": "null" } ], "default": { "buffer": 100 }, "description": "Default configuration for writing header data to a CSV file. If ``None``, use the default params in BufferedCSVWriter. Note that this does *not* control whether header metadata is written during capture, for enabling/disabling, use the ``metadata`` kwarg in the capture method." } }, "$defs": { "CSVWriterConfig": { "description": "Configuration for :class:`miniscope_io.io.BufferedCSVWriter`", "properties": { "buffer": { "default": 100, "description": "Buffer length for CSV writer", "title": "Buffer", "type": "integer" } }, "title": "CSVWriterConfig", "type": "object" }, "StreamPlotterConfig": { "description": "Configuration for :class:`miniscope_io.plots.headers.StreamPlotter`", "properties": { "keys": { "description": "Keys to specify what fields of the given model to plot", "items": { "type": "string" }, "title": "Keys", "type": "array" }, "update_ms": { "default": 1000, "description": "Update rate for stream header plots in milliseconds", "title": "Update Ms", "type": "integer" }, "history": { "default": 500, "description": "Number of stream headers to plot", "title": "History", "type": "integer" } }, "required": [ "keys" ], "title": "StreamPlotterConfig", "type": "object" } } }
- Fields:
- field csvwriter: CSVWriterConfig | None = CSVWriterConfig(buffer=100)¶
Default configuration for writing header data to a CSV file. If
None, use the default params in BufferedCSVWriter. Note that this does not control whether header metadata is written during capture, for enabling/disabling, use themetadatakwarg in the capture method.
- field plot: StreamPlotterConfig | None = StreamPlotterConfig(keys=['timestamp', 'buffer_count', 'frame_buffer_count'], update_ms=1000, history=500)¶
Configuration for plotting header data as it is collected. If
None, use the default params in StreamPlotter. Note that this does not control whether header metadata is plotted during capture, for enabling/disabling, use theshow_metadatakwarg in the capture method