Pololu Robotics & Electronics
Menu
My account Comments or questions? About Pololu Contact Ordering information Distributors

Pololu Forum

VLX53L0X timeout issues

I found a thread on this same issue from two years ago that never seemed to have a conclusion. Rather than bumping the thread, I am starting a new one.

I have three units on a device, but I can see the problem even when I hold two of the units in reset. The issue is that after a random-ish amount of time, the Pololu library readRangeContinuousMillimeters returns -1 (65535). From code inspection this only happens if the RESULT_INTERRUPT_STATUS bit stay low for the time out period. When this happens there is no recovery. It stays stuck from them on. Also the IR light from the sensor stops coming out as seen using night vision goggles.

I have not tried to re-initialize the part when it gets stuck, but I think it would likely come back to life. As this is what happens when I do a reset holding power on to the sensors.

My initialization is to call setTimeout(500), init(), and then startContinuous().

Somehow I think the sensor is losing its programing.

The I2C master is an ESP32. It gets its power either via a USB cable or via battery. The battery architecture is two 18650’s that feed into a Pololu 5 volt buck regulator. The 5V feeds the ESP32 that has its own 3V3 buck to power the board.

I do find the time it takes for the VL53 to enter the "timeout error mode "takes less time (on average) running from the USB. Sample size of about 20.

The wiring is point to point solder with 26 gauge wire of lengths less than 8 inches longs.

The environment can be very benign when the failure occurs. The sensors are mounted in a robot with two drive motors, but the failure can happen even when the motors are disconnected from the system. The reset of the robot includes four reflective light sensors connected to analog pins on the ESP32.

The failure SEEMS to occur if the sensor is looking “into the void” or at a close object.

I am using the Pololu Arduino library with one exception. Since I could not tolerate the busy loop wait in the readRangeContinuousMillimeters routine, I create a polling method to determine if a measurement exists before I call the above to retrieve the new measurement. The function looks like this:
bool VL53L0X::rangeAvailable(void)
{
return (readReg(RESULT_INTERRUPT_STATUS) & 0x07) ? true: false;
}
(By the way, I suggest this as an addition to your library.)

So was any progress ever make in finding out why people’s sensors would stop reporting?

-Skye

Ran the experiment where I re-initialize the sensor if it gets into the timeout error condition. It solves brings the sensor back from the dead. The questions now is why is it dying?

Bad power? I guess I can monitor the power at the sensor with a scope to see.
Bad I2C command? I guess I can monitor the bus with a logic analyzer.
Issue with these sensors?
Overheat? Think not as the re-init works right away.

Right now I am seeing a re-init every minute or so. Sometimes I see a few within just a few seconds. Grrr.

Often I can toggle the X_SHUT signal to reboot the processor and then reprogram it. After than it will work for another period of time. Unfortunately at other times, toggling the X_SHUT signal (20 ms down, 20ms up) before programming does not work. The very first I2c transaction (getting the model number) fails with a NAK. In fact, at this point every things fails with a NAK.

And yet I do not think the bus is hung because the SDA and SDL lines are not stuck.

Try as I might, I have yet to catch the transition from working to failed on a logic analyzer. The event is just to random and I do not have the patience to catch 10 seconds of data at a time hoping to catch the event that might happen up to 20 minutes in the future.

I guess the next solution is to switch the power (3V3) to the sensors to see if that will kick them free.

Heisenberg’s Uncertainty Principle: The act of measuring something changes that something.

I have instrumented the sensor hoping to catch a failure occurring. But it has now been running for 3 hours straight without a failure.

I am wondering if the slight capacitance of the logic analyze probes might be fixing the problem.

Hello, Skye.

Thanks for sharing your findings on this issue. I don’t think we at Pololu ever really looked into it before now (since it doesn’t seem to be a problem on other platforms, including standard 8-bit AVR-based Arduinos), but I tested this today with an ESP32 board and was able to reproduce something like what you described.

What I observed was that even though the library reports a sensor reading of 65535 when it gets stuck, it is not reporting a timeout, meaning it is actually reading 0xFFFF from the sensor over I2C.

In turn, this seems to be caused by some strange behavior that takes place while the library is polling for a new reading to be available. Sometimes, instead of writing just the register address for RESULT_INTERRUPT_STATUS (and then proceeding to read the register value), the ESP32 apparently keeps toggling SCL for a while without driving SDA, effectively writing 0xFF values into all of the VL53L0X’s registers. This goes on for pretty much exactly 50 ms, after which the program tries to continue running normally, but it will then only read 0xFF from the sensor.

If you do manage to confirm that this is the same thing that happens in your setup, that would be useful to know. At this point, I think there’s something wrong with either the ESP32’s I2C hardware or the Wire library implementation for it, but since I’m not aware of any widespread ESP32 I2C issues (do you know of any?), I suspect there’s a specific interaction with something in our library code that is causing this problem.

Kevin

Kevin,

First let me express my admiration that you took the time to actually connect up an ESP32 to the sensor, write some code, and capture my issue. That is truly remarkable. I was expecting more of a “Never seen that one before” type of answer. Kudos.

Second, I am thrilled that you caught something on a scope/logic analyzer. The error happens on my system now so infrequently (sometimes as much as 6 hours when I am looking for the problem) that the chance I catch it in the buffer is small. When I am not looking for the problem (and not instrumented to catch it) it seems to happen every few minutes. Go figure.

I think a runaway I2C peripheral in the ESP32 seems like the probable source of my problem. Armed with this information, I can set up the triggers on my LA to look for an extended I2C transfer. I will try to do that over the weekend and report back. This would explain much but possibly not everything. Once the system does “go crazy” I can connect up a LA and see what is going on. In at least one case, all the reads to the sensor were NAKed. Making me believe that the sensor was not responding at all. In fact, in these cases, I would toggle the shutdown pin to ‘reboot’ the sensor. That did not seem to have any effect. But writing this down, I could see a case where the runaway writes would nuke the I2C address register making the device respond to a different address than I am using. That still does not explain why the ‘reboot’ did not restore the system unless the ‘reboot’ does not reset the internal registers.

Again, you have given me a fantastic clue as to where to look. I have also gotten some logic level FETs on order to fully power cycle the sensor if it goes spastic in the future. Now it is time to Google weird I2C behavior on the ESP32. I would not be surprised if this was as issue since it is a multiple core running an RTOS. A poorly written driver could get corrupted if it was not multithreaded safe.

You folks rock.