The Troubleshooting and Repair for On-board Charger (OBC) Thread

[skylogger]

Just got a phone call from another AEVA Member here in Western Australia that said her charger has also died in
here 2010 I-MIEV.
Looks like these are dropping like flies. She will be sending me a photo of the top of the charger so I can check
the part number and see if its same version as the other ones we have come across so far.
I'll post the details here once I get more info.

 

[kiev]

Did you check the 20A fuse in the MCU? Open the small access cover to see it.

 

[kiev]

i did a little work to tidy up the first post with links for the user names to their initial failure reports.

i also added links in the 2nd post for troubleshooting steps and an index of sorts with links to sketches, schematics, circuit descriptions and discussions, to help find stuff quickly. Thanks to coulomb for the idea to create this feature. Let me know if you would like something added to the index.

misspelling of charrger is on purpose to stop the addition of links by the forum bots whenever that word is used... :evil:

 

[skylogger]

Hi KIEV / Coulomb:

This second charrger that I am now working on, has all of the symptoms that the first charrger had.
The 20 AMP 450V Fuse in the MCU was blown, and the SMD Caps on the vertical pcb in the dog house were blown up.
I've de-soldered the verticle pcb and replaced it with two 1000pf 3kv ceramic through hole caps, which is what the previous version used.
I've visually inspected all of the components on the top controller PCB, and on the bottom pcb for other burnt components, and trying
to spot and poor solder joints. I've particularly looked at the area around the 39k resistors near the flexstrip cable on the main board,
which is what the problem was on the previous charrger that I worked on, but on this one, all the resistor solder joints look good and ohm out ok. From what Electronpusher said, he seems to have a charrger with the same faults this one has.
I've replaced the 20 amp fuse in the MCU In preporation of testing back in the car, but have not put the charrger back into car yet till
I can come up with what the cause is that if blowing up caps and the fuse.

It is interesting that all of these charrgers have the caps on the output filter getting blown up. The 2x 1000pf caps are rated at above 3kv.
This part of the circuit normally does not go above 365 volts, so to take out a 3kv rated cap, it must be something generating a
spike in voltage.I am thinking that in the case of the first charrger that I worked on, The fault with the 39k resistor caused a failure in voltage regulation. The difference between charrger voltage and battery voltage caused the current to be too high and blew the 20 amp fuse.
When the fuse blows, the sudden vlotage chage causes a discharge of the inductors in the filter might be what causes a spike that blows the 2x 1000pf caps.

 

[kiev]

i'm working on jray3's charger which had the same symptoms--blown fuse in the MCU and blown snubber caps.

i'm thinking it is caused by a huge inductive spike when the boost stage switching gets interrupted. There are the two transformers plus the two inductors in the upper right corner.

Check all the little parts around T501 and T502. i haven't traced any of that yet but that will be my next target when i can get back to it.

 

[coulomb]

I've replaced the 20 amp fuse in the MCU In preporation of testing back in the car, but have not put the charger back into car yet till
I can come up with what the cause is that if blowing up caps and the fuse.

Very wise to not just run the charrger with only the fuse and obvious failed capacitors replaced. It would be great to know what the cause is, but the more urgent question for now is what else has blown up. Kiev wisely mentioned parts around the pulse transformers; on the Elcon chargers and Axpert inverter chargers, the gate driver components often also fail. But this might be different, as the switching devices may not have failed.

It is interesting that all of these chargers have the caps on the output filter getting blown up. The 2x 1000pf caps are rated at above 3kv.
This part of the circuit normally does not go above 365 volts, so to take out a 3kv rated cap, it must be something generating a
spike in voltage.I am thinking that in the case of the first charger that I worked on, The fault with the 39k resistor caused a failure in voltage regulation. The difference between charger voltage and battery voltage caused the current to be too high and blew the 20 amp fuse.

My understanding is that these charrgers are all essentially current controlled. They set a current, and the voltage follows; the pack gradually rises in voltage as its state of charge increases. It's not like they can set the voltage, and if too different from the pack voltage, too much current flows. So I'm not totally convinced that your badly soldered 39k resistor was the main cause of your charrger failure.

When the fuse blows, the sudden vlotage chage causes a discharge of the inductors in the filter might be what causes a spike that blows the 2x 1000pf caps.

Yes, there is a huge quantity of energy stored in the inductors, and interrupting the current will make the inductors angry But what is interrupting the charrger output? I've not thought of a mechanism as yet. I suppose that a 39k resistor suddenly going open circuit might do it. Car charrgers do suffer a lot of vibration, so other parts might be shaking loose, or letting go due to heat after being shaken loose earlier with driving.

[ Edit : charger -> charrger, to talk like a pirrrate and to avoid the evil bot-links. Thanks Kiev, for that idea.]

 

[electronpusher]

From what Electronpusher said, he seems to have a charger with the same faults this one has.

Correct, we have to two SMD caps on the riser PCB blown as per your board. We will hopefully have the car moved out to our new house this coming weekend (8/9 and 9/9), we can then start working on our OBC, and inspect the fuses in the MCU and let you all know.

Will be replacing the caps with through the holes (do you have a link where I can buy the caps?).

I have a feeling that the caps that are blowing may be a weak point. I discussed this with my PhD supervisor (Biomedical Engineering/Electrical Engineering), and he suggested the caps may be the problem, and said that you could try putting in caps that have a higher voltage rating? Thoughts on that? Do we know what the original caps were rated at? Actually, reading further in your post skylogger, it appears that is exactly what you have done.

So I need to get 2 x 1000pF caps rated at 3kV.

Additionally, with the 20A fuse in the MCU, do you have a link for the replacement fuse, I want to purchase it before the weekend if possible to be ready for the cars arrival? Although we have not looked at that yet, from what I have read I suspect that it most likely has blown.

 

[kiev]

The opamp went to the rail and the controller shut off the FETs immediately was my guess on how the loose resistor could cause the spike, back in Schema 7

With the twisted sister resistor going open circuit, then the op amp becomes a non-inverting follower with a gain of 1+(39/20) = 2.95, and now the output goes to ~4.72V, nearly to the 5V rail. The diode circuit on the ouput would clip that back down a bit, but the microcontroller would be seeing an abnormally high voltage reading. We don't know what it would do in response, but most likely shut down the FETs in an attempt to lower the HV. It may be a sudden shutdown could cause the snubber capacitors in the potted doghouse to see a massive overvoltage and this may have caused them to blow.

 

[coulomb]

Will be replacing the caps with through the holes (do you have a link where I can buy the caps?).

For Westralians, this should do:

https://www.altronics.com.au/p/r2889-0.001uf-3kv-z5u-5mm-ceramic-capacitor/

[ Edit: the image shows a 10 nF part (0.01 μF) part with capacitance code 103; you want a 1 nF (1000 pF) part with capacitance code 102. It will likely be smaller than the one in the photo. It's common to use one photo for hundreds of different valued parts. ]

However, the lead pitch is 5 mm, and it looks like the PCB hole spacing is some 10 mm (maybe 7.5 mm). [ Edit: but you can just bend the leads to fit the wider hole spacing. ]

If you want to order on-line, since it's such a small order, you'd want someone with free shipping with any sized order, which in Australia means RS-Online:

https://au.rs-online.com/web/p/ceramic-single-layer-capacitors/7167324/

You have to buy a bag of ten, but that's still under $10 including GST and the free shipping. Plenty of spares.

I have a feeling that the caps that are blowing may be a weak point. I discussed this with my PhD supervisor (Biomedical Engineering/Electrical Engineering), and he suggested the caps may be the problem, and said that you could try putting in caps that have a higher voltage rating? Thoughts on that? Do we know what the original caps were rated at? Actually, reading further in your post skylogger, it appears that is exactly what you have done.

So I need to get 2 x 1000pF caps rated at 3kV.

Well, there's no reason that there should be even 2 kV in there, so it's not clear that 3 kV parts will solve the problem. Plus, how are other parts going to react with >2kV spilkes in there? But at least a 3 kV part should be a little sturdier than a 2 kV part.

Maybe a film capacitor would be more appropriate? Much bulkier, though.

Perhaps X rated safety capacitors would be better; they are designed not to explode in extreme over-voltage situations. But they do this by fusing to open circuit. I think that considering their cost, it's better to have these cheap things explode and save other, far more expensive parts. If indeed they can do that.

Additionally, with the 20A fuse in the MCU, do you have a link for the replacement fuse, I want to purchase it before the weekend if possible to be ready for the cars arrival? Although we have not looked at that yet, from what I have read I suspect that it most likely has blown.

That's also a problem. We haven't found an on-line source for these that is drop-in. I mention the nearest one I could find on-line in this post. JRay3 has the Mitsubishi part number in this post. But it seems that it's difficult to source in the USA; it might be a lot harder to find in Australia. But it might be worth a phone call or two. My guess is that it will have to come from Japan, which might take weeks, or months considering other posters' experiences . If that's the case, you may as well wait till you know you need it.

 

[skylogger]

Hi Electropusher:

The Mitsubishi part number for the original fuse is: #9499A656 I bought this from the local Mitsubishi dealership for $24
There was no stock in WA but there is some stock in SA, it it may day a few days to get to your location also.
While I was waiting on the original 450v rated fuse to come in, Just for testing, I used a standard 20 AMP fuse 240v rated in an INLINE
fuse holder that I bought from Altronics. I put half moon ring terminals on the wires of the fuse holder so they would slide under the
mounting screws of the original fuse that had metal tabs. I also kept a fire extinguisher handy in case the fuse holder melts since
the fuse and the holder are not rated to 450v, and I was just doing this during a test ;-)

I used some dentist scraping and picking tools that I bought from the chemist, to dig out the rubbery potting around the vertical PCB in the doghouse. The original caps in the dog house were 10% rated. I could have waited to get the original Murata parts from Element14 or Digikey, but did not want to wait for them to come in, so I bought some 20% caps from Altronics (Altronics did not have 10% ceramic 3kv, only 20%), but I checked a few with a meter while at the stores bin, and matched two that were actually within 10% of 1000pf before I paid for them at the counter at the store. The Altronics part number I bought was R2889
There are pins that come up from the main board that the vertical PCB is mounted to. There are pins on both top and bottom of the vertical PCB, so you have to de-solder 8 connection pins, and ending up with two pins forming a fork coming out of the via of the main board.
I formed the leads of the through hole caps so it would slide between the two pins forming the fork, and soldered them back together.
This way, I could do all of the rework on the top side of the main PCB without even removing the PCB.

 

[coulomb]

Just for testing, I used a standard 20 AMP fuse 240v rated in an INLINE
fuse holder that I bought from Altronics. ... I also kept a fire extinguisher handy in case the fuse holder melts since
the fuse and the holder are not rated to 450v, and I was just doing this during a test ;-)

I really don't recommend doing this, even for a quick test. If that fuse had to interrupt a few kilo-amps, as the battery can easily provide, then the resultant arc won't be extinguished by an AC rated fuse, and I think even a fire extinguisher would not extinguish the arc. An arc tends to be around 4 ohms, so that would be some 30 kW at 350 VDC. This isn't even full power for the motor, so the battery can easily dish this power out for tens of minutes, and the main battery fuse won't blow. You'd have to use insulated wire choppers to stop the arc, and separate the wires by some 350 mm (around 1 mm per volt once an arc starts). You need to do all this with a welder's helmet on so you don't burn your eyes out from all the ultraviolet radiation. Meanwhile, 30 kW where it's not designed to go will cause serious damage to the cabling, and likely start some serious fires.

DC is a LOT harder to break than AC; AC goes through zero 100 or 120 times per second. I'm glad you survived this test; it could have ended up so very much worse.

 

[skylogger]

Hi Coulomb:
Your details regarding extreme situation that can occur across a fuse that is not rated to 450v, probably explains also what is happening
with the 2x 1000pf caps in the dog house that are exploding. The caps are across the same high DC voltages in the output circuit that the fuse has to deal with, So they would also be a weak link where arcing could occur. You mentioned rather long distances that Arcs could jump across, so it would be easy to see why this might be a problem with the small distances on the little vertical PCB and the SMD Caps in the doghouse where the caps are exploding. I see the main PCB has groves cut in it that opto-couplers have to bridge across to avoid the HV Problem on other parts of the design on the main PCB.

 

[jray3]

Just for testing, I used a standard 20 AMP fuse 240v rated in an INLINE
fuse holder that I bought from Altronics. ... I also kept a fire extinguisher handy in case the fuse holder melts since
the fuse and the holder are not rated to 450v, and I was just doing this during a test ;-)

For testing purposes, I used some spare fuses from my solar arrays. They are convenient cartridge mini-fuses rated to 600 VDC and much more commonly available.
https://www.littelfuse.com/industries/solar/600-vdc-fuse-and-fuse-holders.aspx

 

[coulomb]

You mentioned rather long distances that Arcs could jump across, so it would be easy to see why this might be a problem with the small distances on the little vertical PCB and the SMD Caps in the doghouse where the caps are exploding.

The figure of 1 V/mm is after an arc has started. Before the arc starts, it's about 3 000 V/mm in dry air. So the key is not to let the arc start in the first place. With the correct DC rated fuses and other components, the chances of an arc starting should be extremely low.

It's a bit like fire. In your home, you probably have flammable materials way too close to other flammable materials; it's nearly impossible to avoid. But it's fine unless a fire starts. Fires are pretty rare, but when they start, there can be a cascade of events and the whole house can burn down. Sometimes, depending on conditions, it might burn down a neighbour's house as well.

I see the main PCB has groves cut in it that opto-couplers have to bridge across to avoid the HV Problem on other parts of the design on the main PCB.

That's more about dust build-up than sheer distance. You can have a great distance between conductors when the equipment is new, then dust and grime can build up, absorbing moisture, and these can effectively shorten the distance until an arc could start. It makes a huge difference if you have a conformal coating on the PCB; then the dust can build up, but can't do anything bad because the higher voltage conductors are under the insulating coating. That's one reason it's a good idea to spay a repaired board with PCB lacquer. I do this all the time on the Elcon/TC chargers. They come with various coatings that have to be removed so you can work on them. The lacquer replaces the removed coating, as far as dust protection goes.

It's standard practice where you have a safety isolation (e.g. battery pack must be separated from AC mains) to cut a groove in the PCB, so that dust can't build up across the gap. I suspect that some the ratings agencies (UL, etc) insist on this.

 

[skylogger]

Hi KIEV:

As you suggested, I've been checking the small components around T501 And T502
I've noticed that around T501 is the Q502 with Diode D505 Across pins 1 and 3 of the transistor.
I have my meter set to DIODE checking mode, and I am measuring 0.068 in BOTH directions across
Diode D505. This is probably due to being in-circuit with reading across transformer or other low
impedance components, but I thought I'd point it out to you so when you get around to tracing this,
you can let me know if this looks ok or not. I also get this same situation across D507 Which is the
diode across pins 1 and 3 of Q503 which probably drives T502.
Pin 2 of these transistors is measures 0R to TAB, So Pin 2 is probably the collector? That would lead me
to believe that the diodes in question are across base to emitters and not across collector to emitter?

 

[kiev]

It looks like the gate drive command comes from the top board on CN1 pins 30 and 32 up to Q506 and 507 on the primary side of T502 and 501.

These are 2SC5053 NPN, while the transistors on the secondary side Q502, 503 and Q501, 504 (underside of board above the waffle plate) are 2SA1900 PNP.

The diodes may be schottky, marked S4 1D here, maybe 1N5819W, but the markings i found didn''t really match.

The diodes from the base to the emitter do seem like a strange design, but i think they are used to block reverse current flow and provide for faster switch off of the PNP. See this application note from TI, http://www.ti.com/lit/ml/slua618/slua618.pdf ,concerning gate drivers and transformers in fig. 40.

i started a sketch to figure out how the signals are sent to the waffle plate.

Some notes, they use the same circuit twice on the primary side and 4x on the secondary in order to drive the totem pole pairs down in the waffle plate. The notes are messy but the data is there, when i have time i will add the schematic to the waffle plate drawing. My OBC is all torn apart right now so no way to fire it up and make any measurements.

Sorry for the messy notes, it was a rush job...

i started a poll about the OBC. If the 12V aux battery power to the OBC gets interrupted, due to being old or weak, or the DC/DC has a hiccup, or whatever?, then the 16VDC power supply on the top board will get shut off. Obviously this would cause an instant shutdown of the boost gate drivers and leave a bunch of stored inductive energy looking for a return path back home--blowing snubber caps, fuses, and ??? as the voltage spikes. A 5 or 6 year old OEM 12V may seem to be working okay day in and out, but when it hiccups, it will be catastrophic to the rest of the systems that require contactors, relays and gate drivers to be functioning perfectly without interruption.

 

[coulomb]

Kiev,

If I understand your loose theory, you see the sudden collapse of 12 V into the charger as causing a bunch of failures, particularly the doghouse battery capacitors and the 20 A fuse.

I see it more as sequence of at least two failures. First, the excess energy of the inductors causes the doghouse capacitors to short circuit. This has the effect of collapsing the charrger output voltage, which causes battery energy to flow into the new short circuit, so now there is a high current path from the battery through the 20 A fuse, some filter inductors with low impedance, and the doghouse capacitors. The fuse and the capacitor leads seem to both fuse, and in the case of the SMD capacitors, they explode. It seems a bit strange that they both fuse / explode, but I can imagine it. Perhaps the severely overloaded capacitors actually arc over, keeping the current going long enough to blow the 20 A fuse.

I guess I'm saying that I agree with you, but see it more as a two stage thing than a single event.

I hope I'm wrong, but there could also be other cascading failures. But let's see what turns out to be faulty in some more cases. Skylogger had no more failures, but he may have been lucky.

 

[kiev]

Yes i agree with your excellent summary and like the cascading failure scenarios. In the Elcon charrgers the low voltage power supplies are self-created from the AC input, and in the OBC we rely on the 12V Aux battery to generate them. My loosely connected thoughts were that in either case the loss of LVPS results in chaos.

Some more general thoughts come to mind--that there is a 20A fuse inside the OBC and another inside the MCU, but i'm not sure we have ever seen both blown together? There is also a big honking capacitor inside the MCU with a huge amount of energy that could cause damage.

jray3's box that i have here failed like skylogger's [MCU fuse and snubber caps], but i haven't found a loose part that could have initiated the inductive spike yet.

 

[coulomb]

there is a 20A fuse inside the OBC and another inside the MCU, but i'm not sure we have ever seen both blown together?

An excellent point. As far as I can see, the two fuses are in series, but the MCU fuse connects to the charrger and also the DC-DC. But those currents are in opposite directions: charrger into the battery, but DC-DC out of the battery. These somewhat cancel; with a slow charge, they might nearly cancel. So I'd think that the charrger fuse would always carry more current than the MCU fuse.

So the charrger fuse should blow first. Why don't we hear of this? I've not heard of the charrger fuse blowing even once.

Perhaps the charrger fuse is a slow-blow type, compared to the MCU fuse. I can't see why; slow-blow fuses are usually for loads with high inrush currents. Charrgers aren't like that.
[ Edit: then again, on first connecting the charrger to the pack, there are big bus capacitors that need to be charged. There isn't an output relay as there is for Elcon / TC charrgers. I wonder how they handle that? Ah, I suppose it's all part of the pre-charge process. The charrger output capacitor and the motor controller input capacitors would be pre-charged together. So I can't see that as a reason for a slow-blow fuse inside the charrger. ]

[ Edit 2: Interesting that the charrger output would have to be connected to the battery all the time that the car is driving. I wonder if this is bad for the output capacitor and IGBTs. And perhaps sudden battery current demand could cause currents from the charrger capacitors to the battery. But I can't see how this would cause problems when charging. Unless people are in the habit of revving the motor in neutral while charging? Perhaps a sudden 12 V load could cause a sudden rush of current from the charrger output capacitor, which might wear down the fuses over time, and somehow the MCU fuse goes first? But the DC-DC would not cause significant battery voltage changes, unlike the motor controller, which could have step changes of hundreds of amps. I feel like I'm grasping at straws here. ]

[ Edit: "output capacitors" -> "output capacitor" (C115)]

 

[kiev]

Some more general theoreticals:

So if (during charging) the Aux battery hiccups for some reason, then the DC/DC senses a change in current or voltage such that it goes to the rails trying to maintain the 12V buss--resulting in a huge inrush thru the MCU fuse to the DC/DC. The fuse does it's job of fusing, but the OBC now has no path for the inductive energy spike and the snubblers get taken out. If it were only blown snubber caps then repairs would be somewhat easy for anyone that can solder a wire.