For anyone wondering at the context here, more than 900 subpostmasters in the UK (that is, contracted franchisees running small branch post offices) were accused and convicted of supposedly embezzling money, based entirely on faulty software. See the Wikipedia post for more info: https://en.wikipedia.org/wiki/British_Post_Office_scandal
The people responsible are the kind of people who end up running a large broadcaster (ITV) or being appointed Commander of the Order of the British Empire (CBE). The law does not apply to that class.
The more I read of this the more I think it’s the legal system that’s really at fault here. Any developer that says “trust me, the software works” can’t be trusted. I feel like every user of software has experienced this fact. I feel like they should need third party code audits, field testing and generally a much higher burden of proof such to actually rule out software bugs sufficiently to jail someone.
Historically the assumption was that a machine under question probably doesn't work, so the prosecution must show to a jury that it works or else they haven't met the burden, you're innocent because the machine doesn't work.
This probably feels fine for a while, and then you discover you're repeatedly releasing people who are very obviously crooks who did exactly what they were accused of, but you couldn't prove to the satisfaction of a jury that the machine everybody knows works does in fact work. They need offer no evidence that it doesn't work, that's your problem, they walk free.
So this assumption was reversed. But then you get Horizon. Did you steal £18,284.27 from the Post Office? Don't say "No" because the computer says you did, and unless you can prove why it's faulty, the Post Office can rely on that £18,284.27 because a computer said so and under law the assumption is that it's correct.
Clearly neither of these simple assumptions is adequate. That's a problem.
None of which justifies what was done to these people, that's over and above the technical legal problem of how to deliver a just outcome - nobody at POL or ICL/Fujitsu seems to have cared about a just outcome anyway.
> This probably feels fine for a while, and then you discover you're repeatedly releasing people who are very obviously crooks who did exactly what they were accused of, but you couldn't prove to the satisfaction of a jury
I’m personally ok with this. I’ll always err on the side of innocence. It’s just money and relatively small amounts. In the US we’d usually just terminate someone that we suspected of theft. We don’t try to sue them and may not even call the authorities at some of these amounts. We’d reevaluate our hiring processes and criteria if we noticed the problem was chronic and unexplained. But if we did seek out a court case, criminal or civil, the thing is this “expert witness” even on paper can’t be trusted. He built the software and should be considered biased towards saving face and any confidence he has in the software should be taken with a grain of salt. That’s without hearing him speak and purely on his resume. The prosecution would(hopefully) need to introduce other evidences. Like, follow the money type stuff. Unusual purchases and such. It would have to still paint a pretty clear picture to get put in jail I would hope.
It seems this whole judicial body needs to be reevaluated to me if this type of injustice can be tolerated for years and dozens of people being sent to jail.
This assumption goes to everything, not just theft. Cell phone records show you were in the vicinity of the victim's house at the appropriate time, not actually at home watching Netflix? Unless they can prove the cell network is infallible you can expect to be able to rely on the claim you were home watching a show.
Stopped a mile up the road from a hit-and-run, weaving drunkenly at excess speed? No worries they'll need to prove the speed camera can't be wrong, and prove that all the alcohol technology is entirely reliable and couldn't be mistaken. The guy who arrested you can say you weren't driving well, but chances are your lawyer can trick him into asserting you were drunk, which is inadmissible since machines aren't perfect, that means it's a mistrial, you walk.
The problem isn't necessarily that this happens in every single case, but just that it feels so obviously unjust, just as Horizon does. The UK cut a hole in "Double Jeopardy" rules exactly because of such injustice - with double jeopardy you could do a crime, get off on a technicality and then tell anybody who asks that you did the crime, knowing you can't be prosecuted no matter what. Under modern rules while it won't be easy to prosecute you for a crime twice it's no longer impossible, which makes such boasting an obviously bad idea, you set yourself up by effectively confessing.
So I’ll start by prefacing that my comments here aren’t comprehensive solutions. I can’t fully explain some things here. The legal systems many of us live with have been iterated on for centuries and include who knows how many written words. I haven’t even studied them very much. I’m just a citizen that’s familiar with their ways and have a feel for what is acceptable justice and also for when things go wrong and something feels unjust. That’s what this whole thing feels like.
That said, there’s plenty of room for grey area of “proof”. People get convicted all the time on circumstantial evidence. But usually it compounds. Like, the accused lied to police, changed their story, can’t explain their whereabouts, or is evasive. Even with some so called evidence, I could ask “is that really proof?” If hair is found at a murder scene and DNA identifies the source, does that actually prove they murdered someone? You could argue it just proves the hair is that persons and found its way some how to the murder scene. Maybe it fell off in the grocery store and stuck to someone’s shoe or the wind blew it. Who knows but point being this very well may not be enough to convict someone alone. But with some other facts like unable to produce an alibi and oh it turns out the victim is having an affair with suspect’s spouse… and most reasonable people start to feel comfortable putting this person in prison for life based on the entirety of the facts
> Stopped a mile up the road from a hit-and-run, weaving drunkenly at excess speed? No worries they'll need to prove the speed camera can't be wrong, and prove that all the alcohol technology is entirely reliable and couldn't be mistaken. The guy who arrested you can say you weren't driving well, but chances are your lawyer can trick him into asserting you were drunk, which is inadmissible since machines aren't perfect, that means it's a mistrial, you walk.
So, no I don’t think you’d walk. Hit and run is a felony and pretty easy to determine. Did they stop and render aide or alert a first responder? It’s pretty binary and Doesn’t matter if they were drunk. The drunk think increases the charges if the test was positive, but they fled so likely the cops caught up to them well after it happened and alcohol was metabolized. I’ll ignore that though, and assume they were present and got tested. The reliability and accuracy of the machine has seen many orders of magnitude more testing and “peer review” such that if maintenance has been documented and machine has no record of malfunctions it’s clear it’s probably trustworthy. It’s not one guy who built the machine saying “trust me”. The police would likely also take this driver to get a blood sample collected as supporting evidence. If that was done so timely enough to explain any variance in results, then it’s even more solid of a case. If the variance of results can’t be reasonably explained, the field testing machine results would or should be seen as unreliable. (I think most jury’s would see this however there’s a lot of other factors that could weigh in like a past history or alcohol related crime).
Even worse, "trust me, the software I wrote works".
I guess the problem is the defence didn't have an expert witness who had access to the source code and bug database (if they even had one) etc. to even counter that.
Feels a lot like the nonsense forensics that has been in the news fairly recently.
The software vendor and this engineer looks like it should have done more to speak out on the problems, but the real villain here was the Post Office who actually pursued the prosecutions despite knowing from basically the beginning about significant problems.
The vendor can't bring a prosecution, so this looks like an attempt to obscure the real villain to some degree.
This guy’s a real piece of work. Not only did he architect this software, he was a key witness in several postmaster prosecutions. Deserves every ounce of jail time the state can throw at him, which appears to be approximately zero. There is a special place in hell for this kind of extreme arrogance.
Snort. Yeah, so, back in the late 90s, I met the CTO for ICL (later acquired by Fujitsu, but it was basically what was then left of the 'British IBM'), after evaluating their Visual Basic 5-based front-end software for a day or two, at the instigation of Microsoft UK.
VB5 was not an entirely horrific choice for an UI running on Windows touchscreen hardware at the time, but the project was riddled with basic mistakes, like blocking foreground threads with long-running background operations, and, mostly, threading model mismatches between UI and back-end code (which was all C++, but not the good kind -- and oh, it implemented some weird distributed messaging bus, where you could do just about anything, but nothing really worked, especially not if the ISDN-based network was acting up...).
My recommendation was to upgrade to VB6, which made multi-threaded foreign function calls at least reliable-ish, to re-do the entire calling surface of the C++ libraries accordingly, plus to significantly improve documentation, since literally nobody seemed to know which calls did exactly what (as in: which parameters they required and what they returned) and how to handle retries.
The guy literally listened to me for 30 seconds (possibly less!), then turned around, and told his minions to escort me out and "get someone who understands what we're doing here". Well...
I haven’t seen any coverage of technical details of how the buggy software might have made fraud look possible. Does anyone have a link about that?
A priori it seems a little implausible to me that any software could make such a mistake in a way that didn’t make it obvious it was a bug, but I don’t know the extent to which the Post Office might have mishandled the evidence.
1. Fujitsu employees could remotely log in to accounts and alter them without subpostmasters knowing. The Post Office knew this could cause shortfalls but lied about it in court. [1]
2. Money/transactions frequently went 'missing' according to subpostmasters when actions, such as customers lodging or withdrawing money or buying stamps, were not properly replicated to the central database. This was caused by things as trivial as 2 customers using 2 machines at the same time. These complaints started as soon as the system was deployed but was ignored for nearly 2 decades.
Ah, 2 seems like a sensible explanation. I’ve seen 1 mentioned in the press but never seen any details on whether Fujitsu employees ever did this, and why they might have.
They had untracked access to what are essentially bank accounts. No one has said it out loud but the implication is theft. The other possibility is that they were desperately trying to cover up faults - when subpostmasters reported bugged transactions sometimes it would get silently 'fixed' remotely.
Pure hubris. He worked on the system, people are saying that it is fatally flawed and he's just not accepting that possibility because it feels like a criticism of him.
He can't imagine the bug so the most likely explanation is theft.
We see this attitude here all the time, e.g. the people that think they never write bugs, and tools to avoid them like static typing, Rust, tests, etc. are just for lesser mortals.
My personal bit of hubris is feeling like I create interesting bugs in spite of the tools available. Lots of "compliance is not equivalence" lessons to identify and hopefully learn from, for example.
Yeah it sounds unbelievable but there really are. Maybe not many that think they never write bugs, but plenty that think they very rarely make the same mistakes that everyone else makes.
If you search for "don't write bugs" on HN there are hundreds of results where someone has effectively claimed that.
More than one of my bosses: "I don't understand why this is taking so long ... look I implemented it in 10 minutes, here: [link to commit that failed CI with syntax errors]"
Its amazing that someone could find errors they made in their code, "finish" fixing them, and then surmise there could be no further possibility of errors.
If anything this mental state is a sign of an infinitesimal amount of software experience, or maybe a well spring of infinite naivete we could mine to power space faring civilizations.
I find it most often in leads that get promoted a little too quickly and are still in the dangerous “advanced beginner” stage where they know enough to be dangerous but still lack expertise to know they have no expertise.
At what point do you take a pause in the midst of accusing 900 of your most trusted people of high crimes? Is everything on fucking auto-pilot? I have been prosecuted by an automated system before. The State actually owed me and still owes me restitution.
Hopefully that "engineer" gets exposed so we all know who to never work with. There aren't many devs in the world partly responsible for the death or wrongful prosecution of over 900 people and still denying any responsibility but spitting in the face of the victims by claiming they did a good job.
Stuff like this is why some software engineers should have the same legal requirements that real engineers do, with personal liability for negligence and the risk of fines or jail.
> Stuff like this is why some software engineers should have the same legal requirements that real engineers do, with personal liability for negligence and the risk of fines or jail.
That would pretty much immediately put a stop to "move fast an break things," which would be a good thing. Software engineer culture is pretty terrible because quality is optional and a matter of personal preference.
I can’t read the article due to the paywall but is this the incident where software written by some firm accused postal workers of theft and got them fired? Hard to see how the final outcome can be characterized as a good job, unless they’re claiming they implemented the specifications they were given.
Not just fired, convicted of crimes they didn't commit.
He's defending himself as much as the software. He personally gave testimony against some of the convicted postmasters. He's in denial because otherwise he'd have to admit that he was not just wrong, but personally helped send innocent people to prison.
If I wrote such software that drove otherwise healthy people to kill themselves I would consider myself responsible for their deaths. The motivation to deny this would be immense. I still have no sympathy for him; in my mind, I think he killed people.
> Dude, it's just software. The responsibility lies with management 100%.
If you're an engineer, that's not true. The responsibility largely (though not entirely) lies with you. You're the one informing management on whether the system is sound, safe, and fit for purpose. If you don't want that kind of responsibility, don't call yourself an engineer. And definitely don't give expert testimony that your system couldn't possibly be at fault.
They had a responsibility to establish actual standing for accusing and prosecuting someone. Software is irrelevant at that point. If the software tells me to kill someone I'm going to hopefully step back and think shit over.
Also, many of the 900-plus people affected were made to "pay back" "embezzled" money that was never missing in the first place, and more than a few went bankrupt in the process.
It was a national shame of the type that undermined and destroyed the USSR, and Putin and his ilk have robbed Russia and covered up a deep social culture of theft and corruption.
You know, of course, that the invasion column sent to occupy Kiev stalled because tanker drivers sold the fuel and the army stalled. There are many other things we see on youtube/Telegram daily
Over 900 subpostmasters were prosecuted and 236 went to prison, including a pregnant woman whose baby was taken from her, despite no evidence the money she was accused of stealing ever existed in the first place. People lost their businesses and homes, were shunned by their communities and some committed suicide. And the Post Office knew from at least 2003 the software was broken.
