Bad data during stream | LabJack
 

Bad data during stream

5 posts / 0 new
Last post
blindgoat
blindgoat's picture
Bad data during stream

I'm streaming two channels at +-10V resolution 0 at 48 kscans/second, reading every 8 kscans.  For testing purposes I'm driving my LabJack T7Pro from a signal generator and verifying that the labjack input is correct by using an oscilloscope.  The signal is a pseudo-random bit stream, 5 Vpp centered at zero.  The labjack channels are running as single-ended.  I'm using an ethernet connection.  The code is in python.  The labjack is opened at the start of each capture and closed at the end.  It is not being power-cycled.

Most data captures return values in the range 0 to -16000 or so and consists of a few isolated pulses going negative.  Every now and then I get a series which is exactly what I expect.  It seems to be either one or the other, but never a mixture.

The code (python) is too complex to usefully post, but I was hoping the symptom might be familiar or you might have some idea what's going on.  All suggestions gratefully received.

LabJack Support
labjack support's picture
I'm not quite following.  Are

I'm not quite following.  Are you saying you are getting voltages of 0 to -16000.0, which obviously would not be expected?

If you just run our Python stream example, do you see the same problem?

If you have access to Windows and can run LJStreamM.exe, do you see the same problem there?

blindgoat
blindgoat's picture
Normal

Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:8.0pt; mso-para-margin-left:0in; line-height:107%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;} table.MsoTableGrid {mso-style-name:"Table Grid"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-priority:39; mso-style-unhide:no; border:solid windowtext 1.0pt; mso-border-alt:solid windowtext .5pt; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-border-insideh:.5pt solid windowtext; mso-border-insidev:.5pt solid windowtext; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;}

I totally misunderstood the results I was getting.  My apologies.  The data I captured consisted of two distinct classes:

·        The first class was the signal I expected to record.  It was so compressed on the plot that I mistook it for a flat line, being about +- 1.5V.

·        The second class of signal was the constant value -9999.0.  I both misread the value on the plot and failed to recognize the missing-data sentinel.

So the upshot is that the data I captured consisted of good data interspersed with bits of missing data.

I ran another test.  The conditions were:

  • ·        Labjack T7 Pro over wired ethernet
  • ·        Streaming at 20000 scans/second
  • ·        Reading in chunks of 10000 scans
  • ·        Reading two channels: AIN0 and AIN2
  • ·        Tested each input chunk for missing values

·        Program is python code

o   The eStreamRead call was in a separate thread (using the threading module)

o   eStreamStart and eStreamStop were called from the separate thread

§  No calls to LJM were made from outside the thread during the thread’s lifetime

The program read three good chunks and then encountered missing data.  The table below shows the device and LJM backlog values for each read.

  • Note that the scan count is not exactly 10000 because the count actually starts at a trigger event which is detected on AIN2 (while the data are AIN0).  I think this is fine.

Data status

Cumulative

Device

LJM

 

Scans

Backlog

Backlog

All good

9998

3

736

All good

19998

3

464

All good

29998

3

704

Found one or more -9999.0

 

3

373

Any thoughts much appreciated.  Thanks.

(FWIW the stream read stuffs the captured data into a Python Queue.Queue structure which is then read by the main thread.  We have used this particular organization with other input devices (Roland DuoEX usb sound capture, Digilent Analog Discovery) and had no trouble keeping up.  Of course Our Mileage May Vary.)

Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:8.0pt; mso-para-margin-left:0in; line-height:107%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;} table.MsoTableGrid {mso-style-name:"Table Grid"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-priority:39; mso-style-unhide:no; border:solid windowtext 1.0pt; mso-border-alt:solid windowtext .5pt; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-border-insideh:.5pt solid windowtext; mso-border-insidev:.5pt solid windowtext; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;}

blindgoat
blindgoat's picture
I executed an additional test

I executed an additional test of the likelihood of skipped data as a function of acquisition speed.

I hacked the example code stream_sequential_ain.py into a function that encapsulated the demo and a main routine that invoked the demo with different scan rates.  (The code is attached.)  The table below shows the results of one run.  For each scan rate the test lasted for either 60 seconds or until a missing value flag is encountered.

In this application, at least, missing values occur at rates as low as 10K scans/second.  Is that surprising?  Do you expect better performance?

In our application we are trying to capture all of the values for a two-minute period at a scan rate of at least 10K scans/second.  Do you think that's an unreasonable ambition?

Thanks for your continued help.


maximum duration of a single test: 60 seconds
scans/second  total scans  skips
        5000       300000      0
       10000       115000    112
       12500        56250   2382
       15000       180000    166
       17500        43750    654
       20000        20000   3310
       22500        56250   1484
       25000        50000   1612
       30000       210000   2026
       35000        17500   9118
       40000        20000   4162
       45000       157500   4320

LabJack Support
labjack support's picture
The T7 is capable of 100K

The T7 is capable of 100K samples/second in stream mode, but that depends on connection type. Over wired Ethernet the 40K samples/second (2 channels * 20000 scan rate) you are doing should be achievable, and Ethernet is recommended for better efficiency.

The -9999 samples indicate the T7's stream buffer overflowed, autorecovery mode kicked in and the -9999s are the amount of missed samples while autorecovery mode was on. This occurs when the LJM driver was unable to keep up with the stream's sample rate which could be due to application, computer or connection performance.

Take a look at this section of the LJM documentation and see if it helps:

https://labjack.com/support/software/api/ljm/streaming-lots-of-9999-values

Additionally, try your 20K scan rate with 2 channels over a USB connection and see if it still has issues. If so, that may indicate an issue with computer performance and it keeping up. Try bumping your application's priority level up and/or close other processes to see if it helps. If over USB there is not an issue, then you Ethernet connection may be the cause. Try a different Ethernet wire to your router/switch, or a different port on your router/switch.