Stream DAQ

This module is a data acquisition module that captures video streams from Miniscopes based on the Miniscope-SAMD-Framework firmware. The firmware repository will be published in future updates but is currently under development and private.

Command

After installation and customizing configurations if necessary, run the following command in this Git repository to start the data acquisition process:

>>> streamDaq -c path/to/config/yaml/file.yml
Connected to XEM7310-A75
Successfully uploaded /miniscope-io/miniscope_io/devices/selected_bitfile.bit
FrontPanel is supported
[24-06-11T01:40:45] INFO     [miniscope_io.streamDaq.frame] frame: 1570, bits lost: 0                                    stream_daq.py:524
[24-06-11T01:40:46] INFO     [miniscope_io.streamDaq.frame] frame: 1571, bits lost: 0                                    stream_daq.py:524

A window displaying the image transferred from the Miniscope should pop up. Additionally, the indexes of captured frames and their statuses will be logged in the terminal.

Prerequisites

  • Data capture hardware: Opal Kelly XEM7310-A75 FPGA board (connected via USB)

  • Supported Operating Systems: MacOS or Ubuntu PC (To do: specify version)

  • Imaging hardware: Miniscope based on the Miniscope-SAMD-Framework firmware. Hardware modules for feeding in data into the data capture hardware are also needed but these will be specified in future updates.

Configuration

A YAML file is used to configure this module. The configuration needs to match the imaging and data capture hardware for proper operation. This file is used to set up hardware, define data formats, and set data preambles. The contents of this YAML file will be parsed into a model miniscope_io.models.stream, which then configures the Stream DAQ.

Example Configuration

Below is an example configuration YAML file. More examples can be found in miniscope_io.data.config.

# capture device. "OK" (Opal Kelly) or "UART"
device: "OK"

# The configuration bitstream file to upload to the Opal Kelly board. This uploads a Manchester decoder HDL and different bitstream files are required to configure different data rates and bit polarity. This is a binary file synthesized using Vivado, and details for generating this file will be provided in later updates.
bitstream: "USBInterface-6mhz-3v3-INVERSE.bit"

# COM port and baud rate is only required for UART mode
port: null
baudrate: null

# Preamble for each data buffer. This is actually converted to bytes in the StreamDaq but has to be imported as a string because .yaml doesn't support hexadecimal formats.
preamble: 0x12345678

# Image format. StreamDaq will calculate buffer size, etc. based on these parameters
frame_width: 304
frame_height: 304
pix_depth: 8

# Buffer data format. These have to match the firmware value
header_len: 12
buffer_block_length: 40
block_size: 512
num_buffers: 8

# Temporary parameter to handle bit polarity. This is not actual LSB, so it needs to be fixed later.
LSB: True

DAQ For use with FPGA and Uart streaming video sources.

class miniscope_io.stream_daq.StreamDaq(config: StreamDaqConfig, header_fmt: StreamBufferHeaderFormat = StreamBufferHeaderFormat(linked_list=(0, 32), frame_num=(32, 64), buffer_count=(64, 96), frame_buffer_count=(96, 128), timestamp=(192, 224), pixel_count=(224, 256)))

A combined class for configuring and reading frames from a UART and FPGA source. Supported devices and required inputs are described in StreamDaqConfig model documentation. This function’s entry point is the main function, which should be used from the stream_image_capture command installed with the package. Example configuration yaml files are stored in /miniscope-io/config/.

Examples

>>> streamDaq -c path/to/config/yaml/file.yml
Connected to XEM7310-A75
Succesfully uploaded /miniscope-io/miniscope_io/devices/selected_bitfile.bit
FrontPanel is supported

Todo

Example section: add the terminal output when running the script Phil/Takuya - docstrings for stream daq: what devices these correspond to, how to configure them, usage examples, tests

property buffer_npix: List[int]

List of pixels per buffer for a frame

capture(source: Literal['uart', 'fpga'], read_length: int | None = None, video: Path | None = None, video_kwargs: dict | None = None, binary: Path | None = None) → None

Entry point to start frame capture.

Parameters:

  • source (Literal[uart, fpga]) – Device source.

  • read_length (Optional[int], optional) – Passed to _fpga_recv() when source == “fpga”, by default None.

  • video (Path, optional) – If present, a path to an output video file

  • video_kwargs (dict, optional) – kwargs passed to init_video()

  • binary (Path, optional) – Save raw binary directly from okDev to file, if present. Note that binary is captured in append mode, rather than rewriting an existing file.

Raises:

ValueError – If source is not in (“uart”, “fpga”).

init_video(path: Path, fourcc: str = 'Y800', **kwargs: dict) → VideoWriter

Create a parameterized video writer

Parameters:

  • path (pathlib.Path) – Video file to write to

  • fourcc (str) – Fourcc code to use

  • kwargs – passed to cv2.VideoWriter

Returns:

cv2.VideoWriter

property nbuffer_per_fm: int

Number of buffers per frame, computed from buffer_npix

miniscope_io.stream_daq.main() → None