Reply
 
Thread Tools Display Modes
 
Old 02-15-2020, 08:12 PM   #1
Platinum Member
 
Join Date: Nov 2017
Location: Michigan
Posts: 111
Default Lithium Charging: Balmer, Elite, Starlight Solar and a (Long) Tale of Two Failures

We have been ‘assessing’ this ‘incident’ for three months and decided, in view of recent Forum discussions relating to ‘overcharging lithiums’ - - we’d add our 2 cents worth.

November 5, 2019 wasn’t a good day. It started ok . . . a nice sunny, but already too cold, day along the shore of Lake Michigan. We departed generally southbound on a ‘warmth seeking’ adventure of uncertain destination and uncertain duration. Maybe we’d return by Thanksgiving, maybe not. Well, we had the adventure part right.

The first clue that this might not be a normal trip occurred on a routine check of our Nation’s 2nd alternator (with Balmar external regulator) to make sure it was ‘there and working’. It was there, but no indication of ‘working’. We uttered the expected expletive but didn’t otherwise let this disappointment ruin our joy of escape.

The arctic precursor to winter had not yet hit . . . it was a still sunny and reasonably mild early afternoon as we navigated the maze of cones and other obstructions defining yet another of Indianapolis’ construction zones . . . then a reprieve, the entrance ramp to I465 to take us ‘around’ the eastside of Indianapolis. But the reprieve was short-lived. As we hit the accelerator to vault onto the interstate, the RPMs soared, the speed sagged. “What the ****?” (Our second expletive of this yet infant travel adventure.) Yes, the ‘lever’ assured that we should be ‘in gear’ . . . yet, as our forward momentum continued to dwindle . . . we glided to a safe arrival along the shoulder of the interstate.

It not being the point of this missive to ‘re-live’ our total transmission failure other than to comment that it took two days to find, obtain, and install a new transmission. And during this ‘maroonment’ we were able to determine that our Nation’s alternator was ‘fine’, that the problem was the external Balmar regulator - - the output had failed so, despite all other indications that it was working, it was delivering no current to the alternator’s field winding.

Not a problem, a visit to Home Depot produced several hundred feet of small gauge wire and a few switches - - the former to be used as a current limiting resistor to provide a ‘switchable’ field current, making a crude manual voltage regulator.

By the early afternoon of the second day we’d been released to continue south - - and south was now a major objective as the arctic blast was engulfing us. No problem . . . we can make it to Nashville or northern Alabama by early evening . . . surely it can’t be too cold there!

We flipped on our ‘manual regulator’ to verify that we had ‘lots of amps’ and decided to leave it on . . . might as well have a fully charged battery upon arrival.

We were southbound along some lonely stretch of I65. “That’s an odd odor,” I suggested to my wife, VerJean, occupying the copilot’s seat. We both craned our necks looking for the hoped-for external source. None was immediately identified - - but ‘smells’ are not uncommon while traveling so we ruminated only momentarily as we continued toward the promised warmth. “Hmm, smells a bit like nail polish, acetone . . . ,” I offered this time. VerJean concurred. Finally the stench had become too profound to ignore . . . we took the next exit, pulled into the adjacent gas station parking lot and I ventured back into the ProMaster . . .

OMG . . . nothing looked familiar. A 100 amp current was still pumping into the lithium pack whose voltage could no longer be ascertained . . . every voltage measuring instrument (the MidNite Solar Controller, the Elite BMS System and the Magnum Charger/Inverter) had gone off-scale or was screaming “Over-Voltage”. We panicked . . . although we did take a literal second to check the Elite monitor to confirm that each of our 20 cells was reporting a voltage well over 4 volts . . . we were too mentally shaken or confused to register the precise over voltage quantity . . . 4.2? 4.5? 4.9? Our focus/urgency was to immediately terminate all charging. We scrambled to the front to switch our 200' of ‘resistance’ off . . . promptly followed by placing a 150 amp space heater load on the batteries. It didn’t take too long to regain our composure and to bring voltages down from the stratosphere . . . back to numbers we expected and remembered.

We opened our bed compartment to gain partial access to the battery pack, mostly to allow for ventilation . . . the batteries (still covered with a protective ‘lets not short them out’ plywood shield) appeared to be ‘smoking’ . . . a cloud of ‘something’ unpleasant was emanating from under the plywood. The ‘cloud’, as expected, soon abated. But the smell did not. As voltage and other system metrics suggested that we were not on fire, and that the batteries appeared to still be operational, we elected to continue south, but, with every possible orifice (window, vent etc) open. It was a very unpleasant 4 hour descent south - - cold and smelly.

Upon arriving at an appropriate campsite we took a closer look at ‘things’. Interestingly, we’d never seen all 20 cells so closely matched - - they were all within a hundredth volt of one-another. We surmised that our ‘super charge’ had resulted in a ‘super balance’. All looked good . . . then . . . And, fortunately, the smell had abated enough that we could sleep in the ProMaster.

But, at light of day, the first of a series of ‘oddities’ became apparent . . . two cells (of 4) in one string were of lower voltage, and higher temperature, than the remaining 18.

The Elite BMS system includes 20 small printed circuit boards, each literally bolted to the terminals of respective cells. These boards report cell voltages and cell temperatures. The ‘symptoms’ of the two questionable cells were consistent with cells that have entered “balancing mode”, more specifically, where the ‘shunt resistor’ on the printed circuit board is “on”. The ‘shunt current’ drains the cell (lowers its voltage) while raising the cell’s ‘apparent’ temperature (the cell isn’t really any warmer, it’s just the proximity of the heated shunt resistance that fools the PCB into reporting an elevated temperature). We removed the remaining plywood protective cover to confirm. When a cell enters the “balancing mode” (when the shunt resistance is “on”), a small red LED can be seen on the PCB. The two errant cells were indeed in shunt mode, their red LEDs were on. The ‘oddity’ here is that “balancing” should only occur when the pack is being charged and, then, only when the cell reaches 3.55 volts. We weren’t charging the pack and none of the cells were anywhere close to 3.55 volts.

We concluded that we had suffered a failure of two of our PCB’s. This presented a interesting challenge to our continued adventure - - how to keep this string of four cells reasonably balanced where two of the cells were being subjected to a continuous extra ˝ amp discharge. We weren’t certain how long we could make this work, but decided to continue our adventure south.

The Florida panhandle. The arctic blast had followed us . . . it wasn’t cold, but neither was it warm.

But then new ‘oddities’ surfaced with respect to our Elite BMS . . . other cells, randomly, would report “no voltage” and -67 degree temperatures, sometimes in blocks of cells, sometimes individual cells . . . very unpredictable and random. Then, by the end of the day . . . the last 9 cells ‘disappeared’ . . . they weren’t reporting odd voltages and temperatures - - they simply weren’t reporting anything. They had vanished . . . according to the Elite monitor, we had an 11 cell battery pack.

By this time we’d had conversations with Larry of Starlight Solar of Yuma (from whom we had purchased the system) and with an engineer at Elite (Elite is responsible for the design of the BMS attached to imported Chinese cells). No one could explain what was happening, or why things were deteriorating nor would they opine what the status/health of our lithium pack might be.

We formed out own theory . . . the cells had ‘out-gased’ something that ‘attached’ to the PCB’s immediately above the cell vents . . . which “something” was eating our PCB’s and destroying our system. We immediately undertook a two-day forced march back to the cold midwest where, within minutes of arrival, we commenced the disassembly of the PCB’s from the cells.

Elite had suggested various ‘chemicals’ we might use to clean the boards, including chemicals that would remove the conformal coating where, presumably, somehow, after cleaning, we could re-coat the boards? If they were still ‘alive’. Or replace them as necessary.

Our inspection, however, revealed PCB’s that looked “fine”. There was no obvious ‘stuff’ on the boards other than the noted factory applied conformal coating which, in fact, looked normal to us. There was no sign of ‘eating or etching’ of the board surfaces.

We abandoned our plan of chemically bathing the boards, instead, hand washed them under warm water in the kitchen sink. Hit them with the heat gun to dry and reassembled - - all within hours of returning home. Voila - - all 20 cells were now, again, properly reporting. There was no randomness or unexpected ‘drop-outs’ in voltage/temperature readings. And most importantly, the two errant PCB’s in which the shunt resistors had become ‘stuck in shunt mode’ . . . were no longer improperly shunting. Today, after an additional three months, still no further ‘oddities’. We have not replaced a single PCB or battery cell.

The above tale is pregnant with unanswered issues such as: Were the lithium batteries damaged by an overcharge of such significance that the cells ‘out-gassed’ a ‘cloud of stuff’? And, why didn’t the BMS protect the lithium pack from the overcharge in the first instance?

There are different opinions on the function of a BMS. Some utilize their BMS as a means of controlling/regulating the normal ‘on/off’ cycles of their chargers. Tgregg, for example, uses his Victron BMS to directly control the interface/connection between his ProMaster factory alternator and his lithium pack - - when the charge level of his lithium pack reaches 95%, his BMS disconnects the alternator and, conversely, when the SoC drops to a predefined lower level, the BMS reconnects the alternator.

A second line of reasoning is that the BMS ordinarily does not intervene, it serves only as a ‘last line’ of protection against disaster in the case all else fails. This was/is our design philosophy - - we never expect our BMS to ‘pull-the-plug’ on our lithium pack, either due to over or under voltage conditions.

At least on the ‘charging’ side, we have programmed each of our three chargers to limit their charge - - to self regulate and to stop charging well before a dangerous overcharge condition can occur. (On the low voltage/discharge end, our first line of defense is simply that we don’t operate the pack near the complete discharge region and we watch our SoC. But, in reality, the Elite BMS is our first and only line of defense on the discharge side.)

Again, on the charge end, something elsewhere must fail in order for our BMS to ‘step-in’ and cut-off charging. And, as noted, something did fail - - our 2nd alternator (Balmar) regulator. Now in fairness to Balmar, the Balmar failed in the “open-circuit” mode meaning that it was providing no charging. It was our ‘response’ (to the Balmar failure), our temporary fix that resulted in the overcharge condition requiring Elite BMS intervention.

There could be a question of ‘prudence’ - - was it prudent to install a manual charging system that requires personal oversight, that requires the user to turn it off? As a temporary fix we reasoned that, at the very least, we had the Elite BMS ‘back-stopping’ should we fail to perform our monitoring duties.

Now, this raises an interesting question, a question for Davydd possibly: If your Elite BMS is not expected to intervene and, in fact, over a three year period has remained a ‘bystander’, how do you know that it is working? Does one have to deliberately overcharge (smoke) their batteries to confirm its correct operation?

We didn’t test this feature of our system, we assumed it was working. Yet, in hindsight, looking at charge/discharge data we’ve taken each year in running full range battery capacity checks, the evidence was there, at least to an experienced eye. Our Elite BMS was never working. Yes, it was producing all the voltage, temperature and SoC data . . . so all appeared well . . . but in reality its most important raison d’ętre, was never tested and was not working.

Thus, like most disasters (as a pilot, aviation accidents come to mind), it generally takes two or more causes to reach ‘final’ demise - - in our situation, a failed Balmar regulator, a failed Winston ‘observer’, and a failed Elite BMS all conspired to achieve our gaseous conclusion.

So let’s look briefly at the Balmar. There aren’t too many external regulators currently on the market. The Balmar is probably a good one, overall, and adequate. But for those of us who have had to personally ‘learn’ their peculiar system (not just Bulk, Absorption and Float, but its time-based switching algorithm, as well) and, even more annoyingly, have had to program the Balmar using its little ‘magnetic screw driver’ in a precise timed-tapping of a ‘red dot’ as certain cryptic characters scroll past its two 7-segment displays . . . well, it’s a pain. We’re not enamored.

Notwithstanding, we telephoned Balmar to negotiate a repair. Nope. “We don’t repair our regulators,” resounded in our ears. “So what do we do,” we inquired? “Buy a new one,” came the response. And, oh yes, we learned that we could pay full price for it. If a company is not going to support their own equipment, can’t they at least offer a significant discount when one of their customers is forced to buy a new one? Not part of the marketing program of too many companies, it seems. We’re compiling a list of offenders, Wilson, Microsoft and, now, Balmar - - all companies offering no repair/replacement options. (Please excuse our tirade, but we recently discovered that one can pay Microsoft $3,000 for an upscale Surface Pro with accessories and, 1 year and 1 day later be told, if it fails, “to throw it away and buy a new one at full price”. Or, if you want to extend the ‘window of happiness’ to 2 years and 1 day, this can be done for a mere extra $140 in purchase price.

We’re not sure if it was Balmar’s indifference, or our noted dissatisfaction with programming, but we elected to try our own DIY regulator . . . which now gives us easy ‘potentiometer’ adjustment over the charge rate/voltage.

And how about Elite and Starlight Solar? We suppose, coming to them nearly three years after purchase and complaining of a manufacturing defect, we should have expected some skepticism. Larry of Starlight Solar is an ok guy, but when we inquired about the Elite BMS’s failure to protect, rather than providing any insights he noted, “Can’t be . . . the Elite BMS will shut-down under such over-voltage conditions if you wired it correctly.” Not particularly helpful. Larry, we did wire it correctly, and it didn’t shut down. But Larry sold us a new “CPU”, found one of the ‘original designs’ and gave us a good deal. So we think (although we haven’t ‘smoked’ our batteries to verify) this new CPU is working.

Now, how about the question - - are the batteries damaged? The jury remains out . . . as we have seen a slight change in the charging profile at the Elite balancing charge level of 14.2 volts. Yet, we have run two compete charge/discharge cycles and both cycles resulted in a full 500 ah capacity. Considering that these batteries have been in substantially continuous operation for three years, we might have expected less. If there is damage, we haven’t been able to see or measure it.

So, those who contend that the LiFePO4 chemistry is tolerant to significant overcharging, they may be correct.
__________________

__________________
2016 159" High Top DIY ProMaster with 500ah Starlight Solar/Elite LiFePo4, 810watt Kyocera Solar w/MidNite Solar Classic MPPT, Magnum 2812/MMP250-60S Charger/PSW w/remote, Nations 280amp 2nd Alternator with Balmar regulator, NovaCool R4500 12/120v frig, 2 burner TruInduction cookstop, SMEV 8005 sink, FloJet R4426143 pump. No A/C or indoor washroom.
Winston is offline   Reply With Quote
Old 02-15-2020, 09:08 PM   #2
Platinum Member
 
Join Date: Aug 2010
Location: Minnesota
Posts: 8,233
Default

Interesting tale of woe, for sure


In out AGM system we run parallel alternators so both can run lower output on high charge and control them with an Amplepower (Charles) regulator that can run two alternators and also allows setting of two separate outputs by toggle switch, with both alternators seeing the same turndown. Bad is it is no longer available and no alternator temp sense. If we lose the regulator, we lose all charging completely to coach and chassis, so pretty critical for us. I found a small, inexpensive, fixed voltage, regulator from Balmar that had output enough for one alternator and wired the dual alternators each with a plug and put a matching socket on the cheapo regulator. If we lose the Amplepower regulator, we just need to unplug both alternators and plug one of them in to the cheapo one and we get plenty of power to get someplace to fix it right. Takes about 5 minutes to do. It may be something for you to consider to have in the toolbox in the future, as Balmars do seem to fail quite a bit.


I am certainly not an expert on overcharging lithium batteries (I hope nobody has done it enough to be an expert), but if the didn't totally short out and die, they obviously didn't have the catastrophic type failure, but the fact that they gassed something might indicate some kind of outgassing of some part of them that is conductive like anodes or cathodes. Metallic gassing smells horrible and not like the "magic smoke" you normally get from burned up electronics and is sometimes extreme enough to cause the very dreaded arcflash in higher voltage setups as the gassing can be conductive. Have you shortened their life, maybe, but if it is not to bad you may still be fine for a long time as you have a lot of cycles to start with. At one place I worked at we had a contract electrician drop a hammer into a 10,000 amp service entrance, and it totally vaporized the hammer and contacts it hit. Blew him back about 20' but was lucky and had a face shield on and didn't catch fire. It smelled horrible in the mechanical area for weeks after that.
__________________

booster is offline   Reply With Quote
Old 02-16-2020, 09:45 AM   #3
Platinum Member
 
markopolo's Avatar
 
Join Date: Oct 2006
Location: New Brunswick, Canada
Posts: 8,132
Default

Thanks for sharing your story.

It would appear that Over Voltage, Over Current (possibly), & Over Temperature protections all failed to kick in.

Could the fault lie with whatever disconnect relay is used or are you fairly certain that the BMS didn't send the necessary disconnect signal?
__________________
Two bikes on sliding cargo box: https://www.classbforum.com/forums/m...icture206.html & 1997 GMC Savana 6.5L Turbo Diesel Custom Camper Van Specifications: https://www.classbforum.com/forums/f...vana-5864.html
markopolo is offline   Reply With Quote
Old 02-16-2020, 10:41 AM   #4
Platinum Member
 
markopolo's Avatar
 
Join Date: Oct 2006
Location: New Brunswick, Canada
Posts: 8,132
Default

Also, have you done any individual disconnected cell testing or monitoring after the event? I was wondering if they rest at similar voltages or are some way different from others after say 96hrs at rest? It could be useful to know what the cells rest at when not paralleled and also given enough time to settle.
__________________
Two bikes on sliding cargo box: https://www.classbforum.com/forums/m...icture206.html & 1997 GMC Savana 6.5L Turbo Diesel Custom Camper Van Specifications: https://www.classbforum.com/forums/f...vana-5864.html
markopolo is offline   Reply With Quote
Old 02-16-2020, 07:33 PM   #5
Platinum Member
 
Join Date: Nov 2017
Location: Michigan
Posts: 111
Default

Quote:
Originally Posted by markopolo View Post
Could the fault lie with whatever disconnect relay is used or are you fairly certain that the BMS didn't send the necessary disconnect signal?
The BMS output (that powers the solenoids) did not release nor did these solenoids release on subsequent tests where Elite said it should. The solenoids do release when 'de-powered'. Our guess is that the output solid state solenoid drivers are defective - - shorted to the +12 volt power rail.

While we haven't run over/under voltage tests on the new CPU, it does drop the solenoids under certain test conditions (e.g. pulling the plug from the 'daisy chained sense boards') so we're assuming we're now protected.

As an aside, we also installed a Victron 712 so now have 'dueling' SoC's and low and high voltage protection systems.
__________________
2016 159" High Top DIY ProMaster with 500ah Starlight Solar/Elite LiFePo4, 810watt Kyocera Solar w/MidNite Solar Classic MPPT, Magnum 2812/MMP250-60S Charger/PSW w/remote, Nations 280amp 2nd Alternator with Balmar regulator, NovaCool R4500 12/120v frig, 2 burner TruInduction cookstop, SMEV 8005 sink, FloJet R4426143 pump. No A/C or indoor washroom.
Winston is offline   Reply With Quote
Old 02-16-2020, 07:39 PM   #6
Platinum Member
 
Join Date: Nov 2017
Location: Michigan
Posts: 111
Default

Quote:
Originally Posted by booster View Post
At one place I worked at we had a contract electrician drop a hammer into a 10,000 amp service entrance, and it totally vaporized the hammer and contacts it hit.

A member of the PromasterForum (who requested to remain anonymous) dropped a 'spanner' on his lithium pack - - he says it didn't take long to 'melt' it. This is one reason we place electrical tape along our battery power busses and position a plywood cover over the batteries. Of course, knowing Murphy's Law, we'll drop something with 'lots of mhos' (conductive) while working on the batteries when its cover is off!
__________________
2016 159" High Top DIY ProMaster with 500ah Starlight Solar/Elite LiFePo4, 810watt Kyocera Solar w/MidNite Solar Classic MPPT, Magnum 2812/MMP250-60S Charger/PSW w/remote, Nations 280amp 2nd Alternator with Balmar regulator, NovaCool R4500 12/120v frig, 2 burner TruInduction cookstop, SMEV 8005 sink, FloJet R4426143 pump. No A/C or indoor washroom.
Winston is offline   Reply With Quote
Old 02-23-2020, 06:55 PM   #7
New Member
 
Join Date: Aug 2016
Location: nc
Posts: 17
Default

re: dropping spanner on battery and causing short

All tools used around lead or lithium batteries should have ALL outer metallic surfaces (other then the jaws of a wrench or the bit of a screwdriver) insulated so that a short it IMPOSSIBLE!

I use electrical shrink-wrap-tubes on screwdrivers. For wrenches, use electrical tape, or Plastidip Liquid Tape, etc.

Harbor Freight sells several sizes of non-conductive socket-wrench composite ratchets :<https://www.harborfreight.com/12-in-drive-composite-ratchet-62618.html>
As long as your sockets aren't as long as the distance between cells, you don't need to worry about them. Otherwise, insulate the outer service.

The only tool that I've found restricted by being insulated is combo box-end/open-end wrenches. You'll lose one end or the other.
ikanode is offline   Reply With Quote
Old 02-25-2020, 05:08 PM   #8
New Member
 
Join Date: Dec 2019
Location: Minnesota
Posts: 11
Default

Quote:
Originally Posted by ikanode View Post
re: dropping spanner on battery and causing short

All tools used around lead or lithium batteries should have ALL outer metallic surfaces (other then the jaws of a wrench or the bit of a screwdriver) insulated so that a short it IMPOSSIBLE!

I use electrical shrink-wrap-tubes on screwdrivers. For wrenches, use electrical tape, or Plastidip Liquid Tape, etc.

Harbor Freight sells several sizes of non-conductive socket-wrench composite ratchets :<https://www.harborfreight.com/12-in-drive-composite-ratchet-62618.html>
As long as your sockets aren't as long as the distance between cells, you don't need to worry about them. Otherwise, insulate the outer service.

The only tool that I've found restricted by being insulated is combo box-end/open-end wrenches. You'll lose one end or the other.
If you REALLY want a combination wrench, you can buy a SnapOn and heat shrink it. They are much thinner wall and will end up the same size as others.
__________________

skunberg is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


» Featured Campgrounds

Reviews provided by

Powered by vBadvanced CMPS v3.2.3

All times are GMT. The time now is 01:21 AM.


Powered by vBulletin® Version 3.8.8 Beta 4
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
×