Author Topic: Fast reading from sensors  (Read 400 times)

Nick_W

  • Full Member
  • ***
  • Posts: 215
    • View Profile
Fast reading from sensors
« on: January 29, 2017, 06:33:11 pm »
I have been toying with adding an acoustic sensor to the 120 module.

This would require sampling audio data at something like 22khz (44 would be best). Should be no problem for am M4.

So I did some tests. Reading an analog input pin as fast as possible, results in 3.2ms per read. Doing the same with a digital input, gives the same value.

The spec for the chip says the ADC conversion is between 10 and 40 us, so that is not the bottleneck. I'm assuming the mcOs is limiting the data rate by time slicing or something.

Could this be improved by increasing priority of the event somehow? I was not running this in a shared event, but from a shared routine run by the boot() event.

Another thought was to add an external fast adc, using SPI to transfer data. I have one in mind that will transfer 12 bits of data at up to 1Msps.

As conversion is initiated by setting SS low, you have to read 16 bits at a time (then SS goes high, and the cycle continues). Trying this results in a data rate of 16 bits (short) every 1.7 ms.

None of this is fast enough to capture audio data.

If I read 1024 bytes at once via SPI, I get a data rate of 1 byte every 9us, which is fine, however I can only read 16 bits (2 bytes) at a time, and there is some sort of delay between successive SPI reads.

Again, I suspect mcOs of time slicing between various processes. I seem to need some way of making a process atomic, so that it doesn't get time sliced by the OS (obviously you couldn't do this for long), or a way of increasing the priority over some OS tasks. Something like nointerrupts, or a way of toggling SS between so many X bytes.

I'm not looking at speech, more like loud sounds, or specific frequencies to be detected, but I can't sample fast enough to do it. It seems a bit of an overkill to add another microcontroller to do this, and would probably suck battery.

The idea is to use a comparator to detect a large sound transient, this triggers an interrupt on the 120 module, that captures say 1 second of audio, analyzes it for specific frequencies, and triggers a message if it matches a profile.

I know I only have 20k or so of RAM to work with, but so far I can't even capture the data.

Any suggestions?

Share on Facebook Share on Twitter


Nick_W

  • Full Member
  • ***
  • Posts: 215
    • View Profile
Re: Fast reading from sensors
« Reply #1 on: January 30, 2017, 02:39:26 pm »
FYI here is my test code:

Code: [Select]
// Test of timing for reading analog sound signal

Define PinMode Pin8 As AnalogInput
//spi test
Define PinMode Pin5 As DigitalOutput

Class SoundTest
    Shared jdata As Json
    Shared spi1 As ExternalSPI
    Shared numberofsamples As Integer
    Shared mqtt_data As ListOfByte
    Shared mqtt_topic As String
    Shared running As Boolean //crude mutex
   
    Shared Event Boot()
        Lplan.SetMidPowerMode(5000)
        numberofsamples = 128 // a sample is 2 bytes
        jdata = New Json
        spi1 = New ExternalSPI
        mqtt_topic = "MCThings/" + Device.mcUID().ToString("X8") + "/"
        mqtt_data = New ListOfByte
        mqtt_data.Add("Booted")
        Lplan.Publish(mqtt_topic + "Status", mqtt_data)
        //SoundTest.runallTest()
    End Event
   
    Shared Event runtests() RaiseEvent Every 1 Minutes
        If running Then
            Return
        End If
        '        mqtt_data.Clear()
        '        mqtt_data.Add(numberofsamples.ToString)
        '        Lplan.Publish(mqtt_topic + "numsamples", mqtt_data)
        If numberofsamples > 4096 Then
            numberofsamples = 128
        End If
        SoundTest.runallTest()
        numberofsamples *= 2
    End Event
   
    Shared Sub runallTest()
        If running Then
            Return
        End If
        running = True
        SoundTest.testAnalog()
        Thread.Sleep(10000000)
        SoundTest.testSPISingle()
        Thread.Sleep(10000000)
        SoundTest.testSPIContinuous()
        Thread.Sleep(10000000)
        running = False
    End Sub
   
    Shared Function testAnalog() As Nothing
        //test read analog pin
        SoundTest.LEDFlash()
       
        Dim Values As ListOfShort = New ListOfShort
        Dim duration As Integer = Device.GetTimeSpan()
        For count As Integer = 0 To numberofsamples - 1
            Values.Add(Pin8)
        Next
        duration = Device.GetTimeSpan()
        SoundTest.Publish("analog", duration, Values.Count)
    End Function
   
    Shared Sub testSPISingle()
        //test read spi single
        SoundTest.LEDFlash()
       
        Dim Values As ListOfShort = New ListOfShort
        Dim value As Short
        Dim duration As Integer = Device.GetTimeSpan()
        For count As Integer = 0 To numberofsamples - 1
            Values.Add(spi1.Read)
        Next
        duration = Device.GetTimeSpan()
        SoundTest.Publish("spi_single", duration, Values.Count)
       
    End Sub
   
    Shared Sub testSPIContinuous()
        //test spi continuous reads
        SoundTest.LEDFlash()
       
        Dim duration As Integer = Device.GetTimeSpan()
        Dim Bytes As ListOfByte = spi1.Read(numberofsamples * 2)
        duration = Device.GetTimeSpan()
        Dim Values As ListOfShort = New ListOfShort
        Dim value As Short
        '        For count As Integer = 0 To Bytes.Count() - 2 Step 2   //does not work for some reason
        '            value = Bytes.ExtractShort(count)
        '            Values.Add(value)
        '        Next
        Dim count As Integer = 0
        While count < Bytes.Count()
            value = Bytes.ExtractShort(count)
            Values.Add(value)
            count += 2
        End While
        SoundTest.Publish("spi_continuous", duration, Values.Count)
       
    End Sub
   
    Shared Sub Publish(test As String, duration As Integer, samples As Integer)
        Dim uspersample As Float = duration / samples
        mqtt_data.Clear()
        jdata.Clear()
        jdata.Add("test", test)
        jdata.Add("duration_us", duration)
        jdata.Add("samples", samples)
        jdata.Add("us_per_sample", uspersample)
        mqtt_data.Add(jdata.ToString)
        Lplan.Publish(mqtt_topic + "Data", mqtt_data)
    End Sub
   
    Shared Function LEDFlash() As Nothing
        LedGreen = True
        Thread.Sleep(3000)
        LedGreen = False
    End Function
   
End Class

Class ExternalSPI
    //class to read external spi sensor
    Shared sensor As Spi
   
    Public Sub New()
        Pin5 = True
        sensor = Spi.Create(125000, 0, Pin.Pin0, Pin.Pin1, Pin.Pin3, Pin.Pin5) //125KHz spi interface
    End Sub
   
    Public Function Read() As Short
        //read single short value
        Return Read(2).ExtractShort(0)
    End Function
   
    Public Function Read(numbytes As Integer) As ListOfByte
        //read data (number of bytes)
        Dim data As ListOfByte = New ListOfByte
        data.AddElements(numbytes) 'size of data to read
        data = sensor.Transfer(data)
        Return data
    End Function
   
End Class

and here are my results:

Code: [Select]
[I 2017-01-30 15:14:07,373] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 7721,
                                                             "samples": 128,
                                                             "us_per_sample": 60.320312
                                                           }
[I 2017-01-30 15:14:19,119] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_single",
                                                             "duration_us": 34790,
                                                             "samples": 128,
                                                             "us_per_sample": 271.796875
                                                           }
[I 2017-01-30 15:15:37,063] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 34882,
                                                             "samples": 256,
                                                             "us_per_sample": 136.257812
                                                           }
[I 2017-01-30 15:15:47,164] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_single",
                                                             "duration_us": 89203,
                                                             "samples": 256,
                                                             "us_per_sample": 348.449219
                                                           }
[I 2017-01-30 15:16:00,304] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_continuous",
                                                             "duration_us": 32989,
                                                             "samples": 256,
                                                             "us_per_sample": 128.863281
                                                           }
[I 2017-01-30 15:17:07,292] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 125183,
                                                             "samples": 512,
                                                             "us_per_sample": 244.498047
                                                           }
[I 2017-01-30 15:17:17,535] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_single",
                                                             "duration_us": 234772,
                                                             "samples": 512,
                                                             "us_per_sample": 458.539062
                                                           }
[I 2017-01-30 15:17:27,902] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_continuous",
                                                             "duration_us": 65796,
                                                             "samples": 512,
                                                             "us_per_sample": 128.507812
                                                           }
[I 2017-01-30 15:18:38,242] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 457550,
                                                             "samples": 1024,
                                                             "us_per_sample": 446.826172
                                                           }
[I 2017-01-30 15:20:11,731] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 1731659,
                                                             "samples": 2048,
                                                             "us_per_sample": 845.536621
                                                           }
[I 2017-01-30 15:20:23,723] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_single",
                                                             "duration_us": 2169586,
                                                             "samples": 2048,
                                                             "us_per_sample": 1059.368164
                                                           }
[I 2017-01-30 15:20:35,235] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_continuous",
                                                             "duration_us": 262909,
                                                             "samples": 2048,
                                                             "us_per_sample": 128.373535
                                                           }
[I 2017-01-30 15:21:53,454] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 6834015,
                                                             "samples": 4096,
                                                             "us_per_sample": 1668.460693
                                                           }
There are a few publishing's missed, but you get the idea.

As you can see the time per sample (sample is 2 bytes) goes up with the number of samples, except for the continuous spi read test. It seems that the individual reads take place at the expected rate, but after a certain number of reads (or time) there is a delay (probably while the OS does something else). The continuous spi read test does not do this, as I suspect the spi read is atomic, and can't be interrupted. So one long read always takes the same time per byte, but multiple reads gets split up into time sliced chunks by the OS, so overall it slows down by the number of chunks read at a time. The more chunks, the slower it gets.

I was hitting 3.2ms per sample (2 bytes) at 8096 reads.

Any ideas on how to work around this? I know it's an extreme case, but there are probably other circumstances where you don't want successive reads/writes delayed by random amounts. Something like Thread.preventInterrupt(True) guards you could put around time critical code.
« Last Edit: January 30, 2017, 02:44:09 pm by Nick_W »

Nick_W

  • Full Member
  • ***
  • Posts: 215
    • View Profile
Re: Fast reading from sensors
« Reply #2 on: January 30, 2017, 06:54:54 pm »
I have a glimmer of a possibility, I do have an ADC with an I2C interface.

Problems here:

Clock speed on 120 module still doesn't work properly (limited to 250khz). Can we fix this soon? Should go up to 400khz.

I2C.read() functions are supposed to accept integers (according to the documentation), but actually accept bytes. This means you can only read 255 bytes at a time. One function (I2C.read(Integer)) does seem to compile, and says that it reads integer bytes followed by a stop bit, but just returns 0 all the time. (I2C.read(byte,True) works properly).

Even with these limitations, reading 255 bytes at a time at 250khz, I can read at 75us per byte, almost good enough.

If you could fix the speed issue, this would probably work. The read functions are a pain, fixing them would also be helpful, but the I2C speed is the limiting factor now.

Any chance of getting this fixed soon?

Thanks.

mc-John

  • Global Moderator
  • Full Member
  • *****
  • Posts: 212
    • View Profile
Re: Fast reading from sensors
« Reply #3 on: January 31, 2017, 07:28:16 pm »
This is a bit more complex than you would expect. First there are problems with 400KHz and using the 2.4GHz radio. The Radio blocks the TWI just too long for reliable operation at the higest speed.
Secondly to transport more than 255 bytes of memory the system requires specific memory configurations which would limit the amount of usable memory for the rest of the system. We think that more general memory is more important than giving memory to one specific function.

Your previous comments are all based on how fast something can be done. mc-OS and the whole environment are all based on simplified programming model. If you read a pin it does a lot of status and synchronizations checks. This cost performance so the system is not optimized for performance. A good example is reading lots of ADC data. While the processor manual specifies the fastest speed based on a specific way of reading. This requires a lot of setup and then you can read fast. If we would have build this we would not have a simple read analog pin interface and we think that simplicity is more important than performance.