Author Topic: Variables Signed/Unsigned and such  (Read 314 times)

Nick_W

  • Full Member
  • ***
  • Posts: 215
    • View Profile
Re: Variables Signed/Unsigned and such
« on: July 28, 2016, 05:27:43 pm »
OK,

After some testing, I have to say your pressure compensation routine does not work, it gives all sorts of random results. My routine also gives wrong results, but they are consistently wrong (ie it gives a large number which varies slightly with each reading). Yours gives medium positive or negative numbers which do not look like pressure readings, and vary a lot.

Still trying to track down the errors, the lack of a uint32_t makes life very hard! Not being able to do bit-wise arithmetic (because they get converted to Floats) is mind bendingly difficult. Some things just do not have a negative value...

This is my attempt:

Code: [Select]
    Shared Function BME280_compensate_P_int32(adc_P As Integer) As Float
        // Returns pressure in Pa as float. Output value of “96386” equals 96386 Pa = 963.86 hPa
       
        Dim var1 As Integer
        Dim var2 As Integer
        Dim p As Integer
        var1 = (t_fine >> 1) - 64000
        var2 = (((var1 >> 2) * (var1 >> 2)) >> 11) * dig_P6
        var2 = var2 + ((var1 * dig_P5) << 1)
        var2 = (var2 >> 2) + (dig_P4 << 16)
        var1 = (((dig_P3 * (((var1 >> 2) * (var1 >> 2)) >> 13)) >> 3) + ((dig_P2 * var1) >> 1)) >> 18
        var1 = (((32768 + var1)) * dig_P1) >> 15
        If var1 = 0 Then
            Return 0 // avoid exception caused by division by zero
        End If
        p = ((1048576 - adc_P.ZeroExtend()) - (var2 >> 12)) * 3125
        If p < 0x80000000 Then
            p = (p << 1) / var1.ZeroExtend()
        Else
            p = (p / var1.ZeroExtend()) * 2
        End If
        var1 = (dig_P9 * ((((p >> 3) * (p >> 3)) >> 13))) >> 12
        var2 = ((p >> 2) * dig_P8) >> 13
        p = p + ((var1 + var2 + dig_P7) >> 4)
        Return p.ZeroExtend()
    End Function

output is
Code: [Select]
MCThings/70074/Pressure 1201306112.000000

Which varies slightly, but is consistent.

I'll look at it again tomorrow, mind is too boggled to think straight now.