Setup: A Trex commanded by a spektrum compatible receiver set in mixing mode for two motors with only #1 and #2 in use. 12v power supply with a separate 5v receiver power source. BEC jumper disengaged. The T-rex run through standard setup/receiver recognition. Identical setup to about 40+ others I’ve done over the past years but using receivers I’ve only used for about six.
The T-rex works normally forward, backward and turning right at any speeds. When turning left at full speed the unit locks into full power spinning the robot to the left. The T-rex then refuses any command to stop or change direction.
Any thoughts? It is a straight out of the box unit received a few days ago.
I’ve never heard of behavior like that. Is your receiver the kind that continues to broadcast the last pulses it was sending if it loses communication with the transmitter? Is the behavior you’re describing 100% repeatable when turning left at full speed or is it intermittent?
Can you see if you still get the problem with the motors disconnected (you can use the motor indicator LEDs to figure out what the drivers are trying to do)? Can you also monitor the situation with the TReX Configurator program to see what the TReX is measuring on its channels and trying to do with the motors?
The behavior on one speed controller at the client site (the original post) is 100% repeatable when turning left at full power.
We’ve been working with another unit and an identical speed controller all today and after considerable effort have been able to intermittently replicate a similar event. The 12v battery must be fully charged (a less charged battery does not work). The unit must be repeatedly moved left/right, left/right to max amperage draw. At some point rather than the thermal cutout engaging the unit will go to full speed in some direction (we had one event forward and one event turning right) and refuse all commands. Once the unit is locked into moving pulling the signal lines off of the speed controller does not effect the movement. We’re trying to recreate again and get the LED colors, but they did not change when the connection to the receiver was cut.
Peak amperage draw was measured at 19A combined using a Watts Up power analyzer (rated for 100A). The speed controller is fan cooled but does not have an additional heat sink.
Triggering the event can be very quick or very difficult. The quickest trigger was after moving back/forth for an extended period, then letting the unit rest for a minute (still on, but no commands) then restarting the movement which resulted in immediate max speed movement right.
We will try to replicate the problem here. Can you clarify what you mean by left and right? Do you mean both motors spinning in opposite directions or one motor stopped and the other motor turning?
Do you continue to get LED feedback from the unit while it is in this stuck-turning state? For example, if you disconnect the control signals while it is stuck, do the LEDs indicate that it returns to safe-start mode (i.e. green LED flashing rapidly)?
Do you have any non-standard parameter settings? Can you see if increasing the motor acceleration parameter makes the problem go away? Also, do you have any capacitors on your motors for noise suppression? It would be interesting to see if adding caps to your motors or additional capacitance across VIN and GND near the TReX affects this behavior.
By left/right I mean moving from full ahead on motor 1 and full reverse on motor 2 to full reverse on motor 1 and full ahead on motor 2.
When the unit is in stuck-moving it does not return to safe start when the control signals are disconnected. All LEDs seem to work normally for movement then stop changing as soon as the stuck-moving begins. We are trying out two additional receivers from different manufacturers to make sure it’s not some new receiver-speed controller incompatibility or joint issue.
We use the Trex straight from the box but synchronize it with our transmitter/receiver once installed using the standard process. What kind of caps would you recommend (uf rating)?
It sounds like the microcontroller is getting into a bad state somehow, and it could be caused by not having caps on the motors. It would be interesting to see if the problem goes away when using other receivers, but it should be standard practice to put 0.1 uF ceramic capacitors on your motors (rated for at least 50 V). This app note discusses ways to decrease motor noise, including the use of caps across the leads. As for the decoupling capacitor across VIN and GND near the TReX power input, this will have to be at least several hundred uF to make a difference since the TReX already has two 330 uF, 35 V electrolytic bypass caps. In general, the the more capacitance you can get here, the better. I recommend an electrolytic cap rated for at least 35 V.
I checked and we have a single .1uf cap running between the motor leads at the motor. I’ll add the power supply cap as well. Given the current draw at these times I wouldn’t imagine it’s going to do much but anything’s worth a shot. I’ll see if I can hook up a scope to check noise levels too.
Additional test results - replicating the failures:
Yesterday we had tested the robot running for extended times over rough and normal terrain. Everything worked perfectly and we would have certified it as ready to ship out. Just to be sure I demanded we bring it back to the shop, let it sit for 5-10 min and do some final back and forth quick left/right movement. Early in these tests it went to full power lock.
The issue is consistent across multiple brands of receiver. This morning my tech turned it on for the first time of the day, commanded the robot full forward, full back, full forward and it immediately took off. He panicked a bit, grabbed it and shut it off before LED settings could be taken down. After that we’ve been running it back/forth left/right and all around but can’t get it to trigger again.
Additional test results - voltage/amperage tests
The best instruments I have show no amperages even approaching 30A combined peak. Consistent maximum run amperage is about 7A for each motor. They are not consistently run at this high a level, but can peak at 7A for minutes.
We see a significant decrease in voltage measured between power supply and controller from 12v- 13.7v to 6-7v under maximum load. We are assuming this is because of limitations in the ability of our batteries to supply power and inherent resistance in the power supply wires.
Additional test results - deliberate overheating
In the interest of replicating the failure we tried to overheat the unit. We couldn’t do so until we cut off fan cooling to the unit. Under these conditions the thermal cutoff engaged. Even in repeated tests the unit never went to any speed locked condition.
Additional history:
As I mentioned we’ve used this speed controller/vehicle combination without issue quite a few times so we’re trying to figure out what has changed. Our most recent changes were 3-4 vehicles ago: adding a larger amperage battery, replacing a twisted pair of wires carrying main power with a single larger guage wire carrying power and new receivers. We’ve pulled the new receivers and replicated the issue, we’ve run a smaller battery and replicated the issue.
Examination of the controllers:
I took apart the two T-Rex boards to check for signs of any kind of damage. There are very two to three faint square patterns on the top of one of the VNH2SP30 chips. Could this be heat damage? If so could the chips be feeding back damaging higher voltage to the microcontroller? I would think given both motors go to full power the issue cannot be with a single chip unless that chip influences the behavior of the other.
One of the boards we are running off of the same power supply as the TRex uses a 12F683 PIC processor, unfiltered except for a 5v linear regulator and a 0.1uf cap. It interprets receiver signals normally before during and after a lockup.
Additional information - replication of the issue
It seems to be easier to replicate the issue immediately after the unit is turned on - at least on Wednesdays.
I am not sure what you mean by the “12v- 13.7v to 6-7v”, but that definitely does not sound good. Are you saying that when the battery is at 12V, you’re only seeing 6V on the TReX power input? That would be indicative of major wiring problems, and if you recently changed how you are wiring things, that makes it even more of a concern. If instead the battery is dropping to 6V, that would not be good for a charged 12V battery. How are you measuring these values? I am concerned that if you are seeing 6V with just a meter, there could be all kinds of spikes going even lower and out of the TReX operating range. Can you clarify what you mean by that “6-7v” measurement and also let us know how your setup is wired (e.g. pictures or schematic drawing including wire sizes and lengths).
We tested voltage at the battery and at the Trex. The voltage at the battery is constant at 12.07v (measured just with meter) but the T-Rex input drops to 8v (meter and Watts up unit) at full stall. We’re seeing 4 volts which is a giant issue, but seems to be from wire losses. We’re adding additional wire runs to check if this is the case. We’re measuring with a Watts Up power meter from Tenergy (specs: all-battery.com/wattsuprcwat … sion2.aspx)
The schematic is pretty simple basically battery–3’ wire–>switch–>2’wire–>circuit breaker(15A is standard)-1" wire->copper plate junction board -2" wire->Trex.
One note, the issue we’re seeing doesn’t happen at full stall. We are shifting from one direction to the other in our tests, but our customer sees this behavior every time they make a left turn. Our tests are using much greater loads and stresses to try to simulate the situation. We could only simulate it when we replaced our standard 15A breaker with a 30A breaker in the test model in our shop.
The next thing we’re going to try is the scope, testing for a faster larger loss.
Do you see any voltage spikes? In general, what does VIN look like at the TReX when you are rapidly switching between left and right in the way that causes the controller to lock up?
We’ve checked for spikes and found variation when power is applied but no spiking. I haven’t been able to get readings during a failure because of the difficulty in forcing one and the challenge of reading an oscilloscope on a robot while holding probes on a speed controller inside the robot which is moving wildly back and forth as we try to force it to shoot off at full speed in an unpredictable direction. We should have additional equipment to help take measurements later today and I will post results.
Is the current best thought a voltage drop which causes the Trex voltage regulator to drop voltage in turn which causes a brownout in the Trex microprocessor?
Was any firmware update or chip supplier change applied to the Trexes in the past few months?
Can your prop the robot up so it’s not moving around when the motors are running?
We do think the problem is something along the lines of what you said. The chip has built-in brown-out reset protection, but if the noise is really bad, it might not be enough. To me, the wiring sounds like a problem, and it’s hard to believe that there is a 4V DC drop with no additional spikes. Over five feet of wire, plus a circuit breaker and switch, is definitely not optimal, so adding caps at the TReX should help. Is it possible to cut down the wire length by a few feet? What’s the wire gauge?
There have been no hardware changes (though the microcontroller will soon be upgraded in a way that should not matter). The last firmware change was over a year ago to fix a small bug with serial baud rate support.
The lifting the wheels suggestion came a bit too late and the most recent test may have damaged the scope, an attached computer, reorganized a portion of our shop before it smacked my leg. Unfortunately my wild robot catchers head home at 3p. I will rerun the test after I get the scope working again.
We can change the wire routing to eliminate several feet of wiring, but that is a change from the quite a few units that have worked without any problems at all. I will check to see if the unit does the same thing with a very short run directly to a battery. We have added additional wire runs as part of the test which seems to have had no effect. A 12k uf 35v capacitor will be added at the T-rex.
Test results this morning:
Direct connect between battery and speed controller with 1’ of thick wire. Voltage drop about a volt. Speed controller went to full power lock movement almost immediately.
This suggests voltage drops from wiring are not the case but power spikes still could be.
We tested movement on a scope and found no sign of pulses outside the 8-12 volt band. We may need a faster scope or one with more Vpp range. We may need to monitor voltage coming off of the 5v regulator built into the speed controller which is what we will try next.
We did get the unit to go to a locked state at a low speed twice which is the first time locking has happened at other than full speed. Both of these events were with wheels raised from the ground and motors moving without resistance. The units continue to show the behavior primarily when first started after a resting period suggesting that heat is may be actually reducing ocurrances.
Since I suspect this is an issue with the microcontroller locking due to noise, it would be helpful to see what the 5V line looks like on the TReX.
We tried to replicate the problem here and were unsuccessful with a completely stock TReX at 16 V using almost 3’-long power cables. Have you changed any of the TReX’s parameters from their default values? What is the stall current of the motors you are using?
