Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - Nick_W

Pages: 1 2 [3] 4 5 ... 15
31
mc-Gateway / Re: Programming the mc-Gateway
« on: February 05, 2017, 06:28:48 pm »
I do agree with that, it does seem a bit blanket that beacons are published, and you can't turn it on and off, or choose the topic (from the gateway).

I have also seen odd behaviour when you have more than one gateway, ie you get the same data published twice (beacon that is), once from each gateway, and you can't turn that off.

This means for instance that the Rssi gets published twice with two different values, unless you can filter the mqtt data by gateway Id.

So you almost need some middleware that recognizes that it's already received some data from a module, and filters it out, to prevent repeat posting of the same data, yet could update a "last heard from" field, so you know how recent the data is.

If gateways could talk to each other, this would open up a lot more possibilities, but maybe we're asking too much. If the new gateway has an esp32, the programming options would open up, right now I think it's an esp8266, so it's just a single thread to do everything.

32
mc-Gateway / Re: Programming the mc-Gateway
« on: February 05, 2017, 12:39:33 pm »
Well if your deploying 1,000 to 10,000 units, you only need one server, just set up AWS and/or EC2, both support MQTT, and process all the messages at one central location. No hardware/infrastructure required.

It's an extra config step, but then anything you do will be. You could set up a web front end, so that users can config what gets sent to Losant, and what doesn't. Plus it would be optional, you don't have to use the middleware if you don't want.

Just saying there are other solutions.

33
mc-Gateway / Re: Programming the mc-Gateway
« on: February 03, 2017, 09:35:57 am »
But you could run it on almost anything else, like a Pi. Do you not have anything running an MQTT broker?

Best solution would be to have small computer (say a Pi) running an MQTT broker locally, plus other programs (like beacon_decode and my logging program). This receives all the data/beacons etc, then you push from there to your cloud service/Losant/whatever. Then you are in control of exactly what is sent and when. And you can watch it all in real time to see what is happening.

You can run python on almost anything, a Pi zero ($5), or the PC you are using to program your modules.

Best bet is a RPi 3, with built in WiFi, plug it in, and leave it running 24/7 with mosquito MQTT broker running and whatever python programs you like. With SD card, you can set this up for $50 or less (if you don't need WiFi, and can use the built in Ethernet, you can use a Pi2 or cheaper).

Seriously, a Pi zero works, size of 3 modules, with a $10 wifi dongle and SD card, you can set up a whole server for about the cost of 1 module.

I doubt MCThings will implement programming of a $95 gateway, when you can set up a general purpose server for $20.

Just saying...

34
mc-Module / Re: ListofShort.Min() Not working
« on: February 02, 2017, 06:52:08 am »
You're right, the For loop now works. Must have been doing something else wrong.

This is the actual code I was running:

Code: [Select]
        For count As Integer = 0 To TotalBytes.Count- 1 Step 2
            //Dim val As Short = (TotalBytes.ExtractShort(count, Endianness.Big) & RESULT_MASK) >> 2 //optional endian
            Dim val As Short = ((TotalBytes(count) << 8 | TotalBytes(count + 1)) & RESULT_MASK) >> 2
            result.Add(val)
        Next

Which now works correctly, before it was returning a list of 1 entry.

Never mind...

35
mc-Module / Anyone know how ExtractShort() works?
« on: January 31, 2017, 04:29:12 pm »
I have the following code:

Code: [Select]
    Dim count As Integer = 0
    While count < TotalBytes.Count
            //Dim val As Short = (TotalBytes.ExtractShort(count) & RESULT_MASK) >> 2
            Dim val As Short = ((TotalBytes(count) << 8 | TotalBytes(count + 1)) & RESULT_MASK) >> 2
            result.Add(val)
            count += 2
        End While

Where TotalBytes is a ListofByte (and result is a ListofShort). The above code works and extracts the shorts correctly. If I use the commented out version (using ExtractShort), then it does not, the values returned are 1/2 the number of entries in TotalBytes (ie if TotalBytes is a list of 128 bytes, val is 64).

RESULTS_MASK just masks the top 4 bits. The values should be from 0 to 1023 (10 bits). As I say, my manual version works, ExtractShort does not.

Can anyone explain if I'm doing something wrong? I figure count is an offset into the ListofByte, but it doesn't seem to work.

Thanks,

36
mc-Module / ListofShort.Min() Not working
« on: January 31, 2017, 03:37:54 pm »
Just FYI,

I found that ListofShort.Min() returns the same value as ListofShort.Max()

I have a ListofShort called Values, and was publishing Values.Max - Values.Min to see what the swing was. I kept getting 0 as the value.

I wrote a workaround that calculates Values.Min, and now all is well (I'm assuming ListofShort.Max() works, it seems to return believable values).

Just add to the bug list.

Oh and "For a As Integer To b as Step 2" doesn't work either you always just get 1 step. I don't know if other step values work or not, but 2 doesn't.
It's the Step value that is the problem, without this (ie leave Step off), the default (Step 1) works (as long as you want Step 1).

37
mc-Module / Re: Fast reading from sensors
« on: January 30, 2017, 06:54:54 pm »
I have a glimmer of a possibility, I do have an ADC with an I2C interface.

Problems here:

Clock speed on 120 module still doesn't work properly (limited to 250khz). Can we fix this soon? Should go up to 400khz.

I2C.read() functions are supposed to accept integers (according to the documentation), but actually accept bytes. This means you can only read 255 bytes at a time. One function (I2C.read(Integer)) does seem to compile, and says that it reads integer bytes followed by a stop bit, but just returns 0 all the time. (I2C.read(byte,True) works properly).

Even with these limitations, reading 255 bytes at a time at 250khz, I can read at 75us per byte, almost good enough.

If you could fix the speed issue, this would probably work. The read functions are a pain, fixing them would also be helpful, but the I2C speed is the limiting factor now.

Any chance of getting this fixed soon?

Thanks.

38
mc-Module / Re: Fast reading from sensors
« on: January 30, 2017, 02:39:26 pm »
FYI here is my test code:

Code: [Select]
// Test of timing for reading analog sound signal

Define PinMode Pin8 As AnalogInput
//spi test
Define PinMode Pin5 As DigitalOutput

Class SoundTest
    Shared jdata As Json
    Shared spi1 As ExternalSPI
    Shared numberofsamples As Integer
    Shared mqtt_data As ListOfByte
    Shared mqtt_topic As String
    Shared running As Boolean //crude mutex
   
    Shared Event Boot()
        Lplan.SetMidPowerMode(5000)
        numberofsamples = 128 // a sample is 2 bytes
        jdata = New Json
        spi1 = New ExternalSPI
        mqtt_topic = "MCThings/" + Device.mcUID().ToString("X8") + "/"
        mqtt_data = New ListOfByte
        mqtt_data.Add("Booted")
        Lplan.Publish(mqtt_topic + "Status", mqtt_data)
        //SoundTest.runallTest()
    End Event
   
    Shared Event runtests() RaiseEvent Every 1 Minutes
        If running Then
            Return
        End If
        '        mqtt_data.Clear()
        '        mqtt_data.Add(numberofsamples.ToString)
        '        Lplan.Publish(mqtt_topic + "numsamples", mqtt_data)
        If numberofsamples > 4096 Then
            numberofsamples = 128
        End If
        SoundTest.runallTest()
        numberofsamples *= 2
    End Event
   
    Shared Sub runallTest()
        If running Then
            Return
        End If
        running = True
        SoundTest.testAnalog()
        Thread.Sleep(10000000)
        SoundTest.testSPISingle()
        Thread.Sleep(10000000)
        SoundTest.testSPIContinuous()
        Thread.Sleep(10000000)
        running = False
    End Sub
   
    Shared Function testAnalog() As Nothing
        //test read analog pin
        SoundTest.LEDFlash()
       
        Dim Values As ListOfShort = New ListOfShort
        Dim duration As Integer = Device.GetTimeSpan()
        For count As Integer = 0 To numberofsamples - 1
            Values.Add(Pin8)
        Next
        duration = Device.GetTimeSpan()
        SoundTest.Publish("analog", duration, Values.Count)
    End Function
   
    Shared Sub testSPISingle()
        //test read spi single
        SoundTest.LEDFlash()
       
        Dim Values As ListOfShort = New ListOfShort
        Dim value As Short
        Dim duration As Integer = Device.GetTimeSpan()
        For count As Integer = 0 To numberofsamples - 1
            Values.Add(spi1.Read)
        Next
        duration = Device.GetTimeSpan()
        SoundTest.Publish("spi_single", duration, Values.Count)
       
    End Sub
   
    Shared Sub testSPIContinuous()
        //test spi continuous reads
        SoundTest.LEDFlash()
       
        Dim duration As Integer = Device.GetTimeSpan()
        Dim Bytes As ListOfByte = spi1.Read(numberofsamples * 2)
        duration = Device.GetTimeSpan()
        Dim Values As ListOfShort = New ListOfShort
        Dim value As Short
        '        For count As Integer = 0 To Bytes.Count() - 2 Step 2   //does not work for some reason
        '            value = Bytes.ExtractShort(count)
        '            Values.Add(value)
        '        Next
        Dim count As Integer = 0
        While count < Bytes.Count()
            value = Bytes.ExtractShort(count)
            Values.Add(value)
            count += 2
        End While
        SoundTest.Publish("spi_continuous", duration, Values.Count)
       
    End Sub
   
    Shared Sub Publish(test As String, duration As Integer, samples As Integer)
        Dim uspersample As Float = duration / samples
        mqtt_data.Clear()
        jdata.Clear()
        jdata.Add("test", test)
        jdata.Add("duration_us", duration)
        jdata.Add("samples", samples)
        jdata.Add("us_per_sample", uspersample)
        mqtt_data.Add(jdata.ToString)
        Lplan.Publish(mqtt_topic + "Data", mqtt_data)
    End Sub
   
    Shared Function LEDFlash() As Nothing
        LedGreen = True
        Thread.Sleep(3000)
        LedGreen = False
    End Function
   
End Class

Class ExternalSPI
    //class to read external spi sensor
    Shared sensor As Spi
   
    Public Sub New()
        Pin5 = True
        sensor = Spi.Create(125000, 0, Pin.Pin0, Pin.Pin1, Pin.Pin3, Pin.Pin5) //125KHz spi interface
    End Sub
   
    Public Function Read() As Short
        //read single short value
        Return Read(2).ExtractShort(0)
    End Function
   
    Public Function Read(numbytes As Integer) As ListOfByte
        //read data (number of bytes)
        Dim data As ListOfByte = New ListOfByte
        data.AddElements(numbytes) 'size of data to read
        data = sensor.Transfer(data)
        Return data
    End Function
   
End Class

and here are my results:

Code: [Select]
[I 2017-01-30 15:14:07,373] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 7721,
                                                             "samples": 128,
                                                             "us_per_sample": 60.320312
                                                           }
[I 2017-01-30 15:14:19,119] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_single",
                                                             "duration_us": 34790,
                                                             "samples": 128,
                                                             "us_per_sample": 271.796875
                                                           }
[I 2017-01-30 15:15:37,063] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 34882,
                                                             "samples": 256,
                                                             "us_per_sample": 136.257812
                                                           }
[I 2017-01-30 15:15:47,164] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_single",
                                                             "duration_us": 89203,
                                                             "samples": 256,
                                                             "us_per_sample": 348.449219
                                                           }
[I 2017-01-30 15:16:00,304] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_continuous",
                                                             "duration_us": 32989,
                                                             "samples": 256,
                                                             "us_per_sample": 128.863281
                                                           }
[I 2017-01-30 15:17:07,292] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 125183,
                                                             "samples": 512,
                                                             "us_per_sample": 244.498047
                                                           }
[I 2017-01-30 15:17:17,535] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_single",
                                                             "duration_us": 234772,
                                                             "samples": 512,
                                                             "us_per_sample": 458.539062
                                                           }
[I 2017-01-30 15:17:27,902] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_continuous",
                                                             "duration_us": 65796,
                                                             "samples": 512,
                                                             "us_per_sample": 128.507812
                                                           }
[I 2017-01-30 15:18:38,242] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 457550,
                                                             "samples": 1024,
                                                             "us_per_sample": 446.826172
                                                           }
[I 2017-01-30 15:20:11,731] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 1731659,
                                                             "samples": 2048,
                                                             "us_per_sample": 845.536621
                                                           }
[I 2017-01-30 15:20:23,723] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_single",
                                                             "duration_us": 2169586,
                                                             "samples": 2048,
                                                             "us_per_sample": 1059.368164
                                                           }
[I 2017-01-30 15:20:35,235] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "spi_continuous",
                                                             "duration_us": 262909,
                                                             "samples": 2048,
                                                             "us_per_sample": 128.373535
                                                           }
[I 2017-01-30 15:21:53,454] MCThings/00011532/Data       : Decoded JSON:
                                                           {
                                                             "test": "analog",
                                                             "duration_us": 6834015,
                                                             "samples": 4096,
                                                             "us_per_sample": 1668.460693
                                                           }
There are a few publishing's missed, but you get the idea.

As you can see the time per sample (sample is 2 bytes) goes up with the number of samples, except for the continuous spi read test. It seems that the individual reads take place at the expected rate, but after a certain number of reads (or time) there is a delay (probably while the OS does something else). The continuous spi read test does not do this, as I suspect the spi read is atomic, and can't be interrupted. So one long read always takes the same time per byte, but multiple reads gets split up into time sliced chunks by the OS, so overall it slows down by the number of chunks read at a time. The more chunks, the slower it gets.

I was hitting 3.2ms per sample (2 bytes) at 8096 reads.

Any ideas on how to work around this? I know it's an extreme case, but there are probably other circumstances where you don't want successive reads/writes delayed by random amounts. Something like Thread.preventInterrupt(True) guards you could put around time critical code.

39
mc-Module / Re: New firmware - so far so good
« on: January 29, 2017, 06:39:45 pm »
So, everything seems well with the latest firmware (except for long MQTT delays).

My longest running module is up to 17 days with no resets. 120 battery life now seems stable. Of course only time will tell with that...

40
mc-Module / Fast reading from sensors
« on: January 29, 2017, 06:33:11 pm »
I have been toying with adding an acoustic sensor to the 120 module.

This would require sampling audio data at something like 22khz (44 would be best). Should be no problem for am M4.

So I did some tests. Reading an analog input pin as fast as possible, results in 3.2ms per read. Doing the same with a digital input, gives the same value.

The spec for the chip says the ADC conversion is between 10 and 40 us, so that is not the bottleneck. I'm assuming the mcOs is limiting the data rate by time slicing or something.

Could this be improved by increasing priority of the event somehow? I was not running this in a shared event, but from a shared routine run by the boot() event.

Another thought was to add an external fast adc, using SPI to transfer data. I have one in mind that will transfer 12 bits of data at up to 1Msps.

As conversion is initiated by setting SS low, you have to read 16 bits at a time (then SS goes high, and the cycle continues). Trying this results in a data rate of 16 bits (short) every 1.7 ms.

None of this is fast enough to capture audio data.

If I read 1024 bytes at once via SPI, I get a data rate of 1 byte every 9us, which is fine, however I can only read 16 bits (2 bytes) at a time, and there is some sort of delay between successive SPI reads.

Again, I suspect mcOs of time slicing between various processes. I seem to need some way of making a process atomic, so that it doesn't get time sliced by the OS (obviously you couldn't do this for long), or a way of increasing the priority over some OS tasks. Something like nointerrupts, or a way of toggling SS between so many X bytes.

I'm not looking at speech, more like loud sounds, or specific frequencies to be detected, but I can't sample fast enough to do it. It seems a bit of an overkill to add another microcontroller to do this, and would probably suck battery.

The idea is to use a comparator to detect a large sound transient, this triggers an interrupt on the 120 module, that captures say 1 second of audio, analyzes it for specific frequencies, and triggers a message if it matches a profile.

I know I only have 20k or so of RAM to work with, but so far I can't even capture the data.

Any suggestions?

41
mc-Gateway / Re: Programming the mc-Gateway
« on: January 29, 2017, 06:08:16 pm »
Or you could use my beacon_decode.py program, as this is exactly what it does.

42
mc-Module / Convert String to Integer
« on: January 26, 2017, 03:03:52 pm »
I'm trying to convert a string to an Integer

Page 91 of the guide gives this function for object type String:

Code: [Select]
Public Function ToInteger(Hex As String) as Integer

But I can't get it to work ("Undefined Function"), and I don't know what Hex is supposed to be (some kind of formatting string I assume).

Im using

Code: [Select]
Integer.TryConvert(motionStopTime, stopInt)

Instead. Anyone know the syntax for converting a String to an Integer?

Thanks,

43
mc-Demo205 / RTC update via GPS
« on: January 24, 2017, 09:38:00 pm »
Does the RTC on the Demo 205 get updated after a GPS fix? if not, is there a way to manually do this? How would a device in the field get its RTC updated, especially when changing batteries?

Also, can the GNSS give altitude information? With a 4 satellite fix? Just curious.

Thanks.

44
mc-Things General Discussion / Re: beacon_decode.py issue
« on: January 24, 2017, 02:33:40 pm »
OK,

Just pushed new version of beacon_decode.py which allows the setting of two different brokers, one the source, and one the destination. Destination in optional, program will always publish back to the source, but now you can optionally publish to a second broker, defined using command line options for:

destbroker
destport
destuser
destpassword
destclientid

Hope this helps the people wanting to decode/receive locally, but publish to the cloud.

45
mc-Things General Discussion / Re: beacon_decode.py issue
« on: January 24, 2017, 01:41:15 pm »
The beacon_decode.py program actually rebroadcasts the decoded beacons via MQTT. The log is just so that you can see what is going on.

If you look at the command line options, one of them is --topic (-t) which is the MQTT topic that the decoded beacons are sent to. Default is MCThings/

So each beacon is received, decoded and published to MCThings/<beacond_id>/<item> where <beacon id> is the same as what was received, and <item> is what the value was decoded as (Temperature, BatteryVoltage, Humidity etc.).

You can change --topic to whatever you want on the command line. If you leave it as default and subscribe on your computer to MCThings/# you should see all your modules data being published (decoded from beacons). To publish to the cloud, add this to the command line:

Code: [Select]
beacon_decode.py -t cloud_topic_whatever/whatever/whatever/

Where cloud_topic_whatever is where you publish things to see them on your phone. You can specify other command line options for broker, port, user_id and password - this assumes that you are receiving and sending to the same broker. If they are different, the program would need to be modified to have two mqtt brokers, ports, users, passwords etc. Easy enough to do, but I was assuming you would just use the one broker.

Data then shows up on  cloud_topic_whatever/whatever/whatever/<beacond_id>/<item> as previously described.


The log is just so that you can see what is happening.

Pages: 1 2 [3] 4 5 ... 15