Join Date: Nov 2017
Lithium Charging: Balmer, Elite, Starlight Solar and a (Long) Tale of Two Failures
We have been ‘assessing’ this ‘incident’ for three months and decided, in view of recent Forum discussions relating to ‘overcharging lithiums’ - - we’d add our 2 cents worth.
November 5, 2019 wasn’t a good day. It started ok . . . a nice sunny, but already too cold, day along the shore of Lake Michigan. We departed generally southbound on a ‘warmth seeking’ adventure of uncertain destination and uncertain duration. Maybe we’d return by Thanksgiving, maybe not. Well, we had the adventure part right.
The first clue that this might not be a normal trip occurred on a routine check of our Nation’s 2nd alternator (with Balmar external regulator) to make sure it was ‘there and working’. It was there, but no indication of ‘working’. We uttered the expected expletive but didn’t otherwise let this disappointment ruin our joy of escape.
The arctic precursor to winter had not yet hit . . . it was a still sunny and reasonably mild early afternoon as we navigated the maze of cones and other obstructions defining yet another of Indianapolis’ construction zones . . . then a reprieve, the entrance ramp to I465 to take us ‘around’ the eastside of Indianapolis. But the reprieve was short-lived. As we hit the accelerator to vault onto the interstate, the RPMs soared, the speed sagged. “What the ****?” (Our second expletive of this yet infant travel adventure.) Yes, the ‘lever’ assured that we should be ‘in gear’ . . . yet, as our forward momentum continued to dwindle . . . we glided to a safe arrival along the shoulder of the interstate.
It not being the point of this missive to ‘re-live’ our total transmission failure other than to comment that it took two days to find, obtain, and install a new transmission. And during this ‘maroonment’ we were able to determine that our Nation’s alternator was ‘fine’, that the problem was the external Balmar regulator - - the output had failed so, despite all other indications that it was working, it was delivering no current to the alternator’s field winding.
Not a problem, a visit to Home Depot produced several hundred feet of small gauge wire and a few switches - - the former to be used as a current limiting resistor to provide a ‘switchable’ field current, making a crude manual voltage regulator.
By the early afternoon of the second day we’d been released to continue south - - and south was now a major objective as the arctic blast was engulfing us. No problem . . . we can make it to Nashville or northern Alabama by early evening . . . surely it can’t be too cold there!
We flipped on our ‘manual regulator’ to verify that we had ‘lots of amps’ and decided to leave it on . . . might as well have a fully charged battery upon arrival.
We were southbound along some lonely stretch of I65. “That’s an odd odor,” I suggested to my wife, VerJean, occupying the copilot’s seat. We both craned our necks looking for the hoped-for external source. None was immediately identified - - but ‘smells’ are not uncommon while traveling so we ruminated only momentarily as we continued toward the promised warmth. “Hmm, smells a bit like nail polish, acetone . . . ,” I offered this time. VerJean concurred. Finally the stench had become too profound to ignore . . . we took the next exit, pulled into the adjacent gas station parking lot and I ventured back into the ProMaster . . .
OMG . . . nothing looked familiar. A 100 amp current was still pumping into the lithium pack whose voltage could no longer be ascertained . . . every voltage measuring instrument (the MidNite Solar Controller, the Elite BMS System and the Magnum Charger/Inverter) had gone off-scale or was screaming “Over-Voltage”. We panicked . . . although we did take a literal second to check the Elite monitor to confirm that each of our 20 cells was reporting a voltage well over 4 volts . . . we were too mentally shaken or confused to register the precise over voltage quantity . . . 4.2? 4.5? 4.9? Our focus/urgency was to immediately terminate all charging. We scrambled to the front to switch our 200' of ‘resistance’ off . . . promptly followed by placing a 150 amp space heater load on the batteries. It didn’t take too long to regain our composure and to bring voltages down from the stratosphere . . . back to numbers we expected and remembered.
We opened our bed compartment to gain partial access to the battery pack, mostly to allow for ventilation . . . the batteries (still covered with a protective ‘lets not short them out’ plywood shield) appeared to be ‘smoking’ . . . a cloud of ‘something’ unpleasant was emanating from under the plywood. The ‘cloud’, as expected, soon abated. But the smell did not. As voltage and other system metrics suggested that we were not on fire, and that the batteries appeared to still be operational, we elected to continue south, but, with every possible orifice (window, vent etc) open. It was a very unpleasant 4 hour descent south - - cold and smelly.
Upon arriving at an appropriate campsite we took a closer look at ‘things’. Interestingly, we’d never seen all 20 cells so closely matched - - they were all within a hundredth volt of one-another. We surmised that our ‘super charge’ had resulted in a ‘super balance’. All looked good . . . then . . . And, fortunately, the smell had abated enough that we could sleep in the ProMaster.
But, at light of day, the first of a series of ‘oddities’ became apparent . . . two cells (of 4) in one string were of lower voltage, and higher temperature, than the remaining 18.
The Elite BMS system includes 20 small printed circuit boards, each literally bolted to the terminals of respective cells. These boards report cell voltages and cell temperatures. The ‘symptoms’ of the two questionable cells were consistent with cells that have entered “balancing mode”, more specifically, where the ‘shunt resistor’ on the printed circuit board is “on”. The ‘shunt current’ drains the cell (lowers its voltage) while raising the cell’s ‘apparent’ temperature (the cell isn’t really any warmer, it’s just the proximity of the heated shunt resistance that fools the PCB into reporting an elevated temperature). We removed the remaining plywood protective cover to confirm. When a cell enters the “balancing mode” (when the shunt resistance is “on”), a small red LED can be seen on the PCB. The two errant cells were indeed in shunt mode, their red LEDs were on. The ‘oddity’ here is that “balancing” should only occur when the pack is being charged and, then, only when the cell reaches 3.55 volts. We weren’t charging the pack and none of the cells were anywhere close to 3.55 volts.
We concluded that we had suffered a failure of two of our PCB’s. This presented a interesting challenge to our continued adventure - - how to keep this string of four cells reasonably balanced where two of the cells were being subjected to a continuous extra ˝ amp discharge. We weren’t certain how long we could make this work, but decided to continue our adventure south.
The Florida panhandle. The arctic blast had followed us . . . it wasn’t cold, but neither was it warm.
But then new ‘oddities’ surfaced with respect to our Elite BMS . . . other cells, randomly, would report “no voltage” and -67 degree temperatures, sometimes in blocks of cells, sometimes individual cells . . . very unpredictable and random. Then, by the end of the day . . . the last 9 cells ‘disappeared’ . . . they weren’t reporting odd voltages and temperatures - - they simply weren’t reporting anything. They had vanished . . . according to the Elite monitor, we had an 11 cell battery pack.
By this time we’d had conversations with Larry of Starlight Solar of Yuma (from whom we had purchased the system) and with an engineer at Elite (Elite is responsible for the design of the BMS attached to imported Chinese cells). No one could explain what was happening, or why things were deteriorating nor would they opine what the status/health of our lithium pack might be.
We formed out own theory . . . the cells had ‘out-gased’ something that ‘attached’ to the PCB’s immediately above the cell vents . . . which “something” was eating our PCB’s and destroying our system. We immediately undertook a two-day forced march back to the cold midwest where, within minutes of arrival, we commenced the disassembly of the PCB’s from the cells.
Elite had suggested various ‘chemicals’ we might use to clean the boards, including chemicals that would remove the conformal coating where, presumably, somehow, after cleaning, we could re-coat the boards? If they were still ‘alive’. Or replace them as necessary.
Our inspection, however, revealed PCB’s that looked “fine”. There was no obvious ‘stuff’ on the boards other than the noted factory applied conformal coating which, in fact, looked normal to us. There was no sign of ‘eating or etching’ of the board surfaces.
We abandoned our plan of chemically bathing the boards, instead, hand washed them under warm water in the kitchen sink. Hit them with the heat gun to dry and reassembled - - all within hours of returning home. Voila - - all 20 cells were now, again, properly reporting. There was no randomness or unexpected ‘drop-outs’ in voltage/temperature readings. And most importantly, the two errant PCB’s in which the shunt resistors had become ‘stuck in shunt mode’ . . . were no longer improperly shunting. Today, after an additional three months, still no further ‘oddities’. We have not replaced a single PCB or battery cell.
The above tale is pregnant with unanswered issues such as: Were the lithium batteries damaged by an overcharge of such significance that the cells ‘out-gassed’ a ‘cloud of stuff’? And, why didn’t the BMS protect the lithium pack from the overcharge in the first instance?
There are different opinions on the function of a BMS. Some utilize their BMS as a means of controlling/regulating the normal ‘on/off’ cycles of their chargers. Tgregg, for example, uses his Victron BMS to directly control the interface/connection between his ProMaster factory alternator and his lithium pack - - when the charge level of his lithium pack reaches 95%, his BMS disconnects the alternator and, conversely, when the SoC drops to a predefined lower level, the BMS reconnects the alternator.
A second line of reasoning is that the BMS ordinarily does not intervene, it serves only as a ‘last line’ of protection against disaster in the case all else fails. This was/is our design philosophy - - we never expect our BMS to ‘pull-the-plug’ on our lithium pack, either due to over or under voltage conditions.
At least on the ‘charging’ side, we have programmed each of our three chargers to limit their charge - - to self regulate and to stop charging well before a dangerous overcharge condition can occur. (On the low voltage/discharge end, our first line of defense is simply that we don’t operate the pack near the complete discharge region and we watch our SoC. But, in reality, the Elite BMS is our first and only line of defense on the discharge side.)
Again, on the charge end, something elsewhere must fail in order for our BMS to ‘step-in’ and cut-off charging. And, as noted, something did fail - - our 2nd alternator (Balmar) regulator. Now in fairness to Balmar, the Balmar failed in the “open-circuit” mode meaning that it was providing no charging. It was our ‘response’ (to the Balmar failure), our temporary fix that resulted in the overcharge condition requiring Elite BMS intervention.
There could be a question of ‘prudence’ - - was it prudent to install a manual charging system that requires personal oversight, that requires the user to turn it off? As a temporary fix we reasoned that, at the very least, we had the Elite BMS ‘back-stopping’ should we fail to perform our monitoring duties.
Now, this raises an interesting question, a question for Davydd possibly: If your Elite BMS is not expected to intervene and, in fact, over a three year period has remained a ‘bystander’, how do you know that it is working? Does one have to deliberately overcharge (smoke) their batteries to confirm its correct operation?
We didn’t test this feature of our system, we assumed it was working. Yet, in hindsight, looking at charge/discharge data we’ve taken each year in running full range battery capacity checks, the evidence was there, at least to an experienced eye. Our Elite BMS was never working. Yes, it was producing all the voltage, temperature and SoC data . . . so all appeared well . . . but in reality its most important raison d’ętre, was never tested and was not working.
Thus, like most disasters (as a pilot, aviation accidents come to mind), it generally takes two or more causes to reach ‘final’ demise - - in our situation, a failed Balmar regulator, a failed Winston ‘observer’, and a failed Elite BMS all conspired to achieve our gaseous conclusion.
So let’s look briefly at the Balmar. There aren’t too many external regulators currently on the market. The Balmar is probably a good one, overall, and adequate. But for those of us who have had to personally ‘learn’ their peculiar system (not just Bulk, Absorption and Float, but its time-based switching algorithm, as well) and, even more annoyingly, have had to program the Balmar using its little ‘magnetic screw driver’ in a precise timed-tapping of a ‘red dot’ as certain cryptic characters scroll past its two 7-segment displays . . . well, it’s a pain. We’re not enamored.
Notwithstanding, we telephoned Balmar to negotiate a repair. Nope. “We don’t repair our regulators,” resounded in our ears. “So what do we do,” we inquired? “Buy a new one,” came the response. And, oh yes, we learned that we could pay full price for it. If a company is not going to support their own equipment, can’t they at least offer a significant discount when one of their customers is forced to buy a new one? Not part of the marketing program of too many companies, it seems. We’re compiling a list of offenders, Wilson, Microsoft and, now, Balmar - - all companies offering no repair/replacement options. (Please excuse our tirade, but we recently discovered that one can pay Microsoft $3,000 for an upscale Surface Pro with accessories and, 1 year and 1 day later be told, if it fails, “to throw it away and buy a new one at full price”. Or, if you want to extend the ‘window of happiness’ to 2 years and 1 day, this can be done for a mere extra $140 in purchase price.
We’re not sure if it was Balmar’s indifference, or our noted dissatisfaction with programming, but we elected to try our own DIY regulator . . . which now gives us easy ‘potentiometer’ adjustment over the charge rate/voltage.
And how about Elite and Starlight Solar? We suppose, coming to them nearly three years after purchase and complaining of a manufacturing defect, we should have expected some skepticism. Larry of Starlight Solar is an ok guy, but when we inquired about the Elite BMS’s failure to protect, rather than providing any insights he noted, “Can’t be . . . the Elite BMS will shut-down under such over-voltage conditions if you wired it correctly.” Not particularly helpful. Larry, we did wire it correctly, and it didn’t shut down. But Larry sold us a new “CPU”, found one of the ‘original designs’ and gave us a good deal. So we think (although we haven’t ‘smoked’ our batteries to verify) this new CPU is working.
Now, how about the question - - are the batteries damaged? The jury remains out . . . as we have seen a slight change in the charging profile at the Elite balancing charge level of 14.2 volts. Yet, we have run two compete charge/discharge cycles and both cycles resulted in a full 500 ah capacity. Considering that these batteries have been in substantially continuous operation for three years, we might have expected less. If there is damage, we haven’t been able to see or measure it.
So, those who contend that the LiFePO4 chemistry is tolerant to significant overcharging, they may be correct.
2016 159" High Top DIY ProMaster with 500ah Starlight Solar/Elite LiFePo4, 810watt Kyocera Solar w/MidNite Solar Classic MPPT, Magnum 2812/MMP250-60S Charger/PSW w/remote, Nations 280amp 2nd Alternator with DIY [formerly, Balmar] regulator, NovaCool R4500 12/120v frig, 2 burner TruInduction cookstop, SMEV 8005 sink, FloJet R4426143 pump. No A/C or indoor washroom.