Sunday 15 October 2023

Fusion Days

The Prince of Wales discusses fusion energy at UKAEA - Culham Centre for  Fusion Energy

 

I recently came across this article on the BBC website commemorating the end of Fusion experiments at JET fusion laboratory

This hit me hard, since it was only a mere 40 years ago, that I, a fresh-faced Physics student, entered the gatehouse at Culham, Oxfordshire to undertake my one and only contribution to British science as I started a 6-month placement at Culham Fusion laboratories.

In truth, it was an exciting time. When I was in the last year of A-levels, me and a few others on the Physics course at Sutton Coldfield College of Further Education had arranged a tour of the site. To be honest, I remember little of it now, but something must have stuck, and while fellow students at Sheffield Polytechnic were going to placements at such dry board manufacturers (where really they were used as cheap labour) I felt I was doing what I really wanted. Real cutting edge physics that would shape the future. My first step to the Nobel Prize would start here.... 

The truth was a little more prosaic. 

The Culham site, was situated in leafy fields just outside Abingdon, consisted of not one, but two organizations. 

JET or the Joint European Torus, was a multinational laboratory bringing the best brains in European fusion research onto one site in an effort to bring to fruition the promise of limitless energy by joining hydrogen to helium. It was based in a gleaming glass building, accessed via a transparent archway bridge. It was new, impressive, and was for many years the cutting edge of nuclear technology, still holding the record for a sustained fusion reaction.

However, that was not to be my home. I was based in the UK fusion labs. Unlike JET, this was based in old buildings dating back to the 40's or earlier. It consisted ofa  ramshackle collection of sheds and offices which was less designed and more evolved into some form of amorphous shape plonked onto the Oxfordshire plains. 

While on the same site, and performing the same function, there was little interaction between each area. While both were aware of their existence, each kept to their own side  (apart from in the cafeteria where each would mingle). In the 6 months I was there, I only went into the JET side once to ask for some postcards. The famed JET torus, and the rest, was always off limits for me.

The UK side was more accessible. My "office" for the want of a better word was an old wooden desk in a storage room, where the accumulated junk of many years had been dumped. I spent many a happy hour, delving through old wooden boxes, to see what treasures lay in them. Generally they consisted of old pressure sensors and various pieces of metal of unknown origin. Nor was this unique. You would often wander around and find buildings with old forgotten stores, like they had abandoned like some Saxon hoard, to be recovered, but the original owner never returning.

Computermuseum Mannheim - DIGITAL DEC

 As well as store rooms, there were other areas of treasure for a young 20 year old. At a time before the first PC's were available, there were rows of PDP-11's mini-computers together with the racks of giant disks that you plugged into the top with a satisfying clunk. These were used to control experiments, gather data and do calculations. While less power than my present mobile phone, they were a great tool to practice programming on, and I would often play around with them generating  waveforms on crude CRT displays attached to them.

And of course there were the experiments. Unlike the JET site whose purpose to drive Fusion physics to its final form, the Culham site purpose was more to explore the edge cases of other approaches. There were a number of experimental models, but unlike the JET pristine engineering, these were more put together with string and gaffer tape. 

The experiment I was associated was called Reverse Field Pinch. The idea being, as much as I could understand it, was a form of magnetic containment that would take less power to achieve the plasma densities required. In order to simulate it, I had a large fibreglass model of a tokamak where we could put low voltage fields through and measure the results. Of course now, you could probably simulate the entire thing on a raspberry pi using Mathcad, but the days of cheap computing power were some way ahead, and instead I had built my 1st electronic circuit running multiple 555 timers to try and simulate the plasma field, while the results could be picked off by two coils attached to oscilloscopes. I was very proud of the electronics, but today would look terrible primative.

One thing I do remember were the rooms with rows of capacitors required to drive the high voltage to generate the plasma required. While I was there, one which consisted of jars of castor oil, exploded with obvious fun results. In fact, one of the offshoots of the Culham work was to use the excess high voltage capacity to simulate lightening strikes on aircraft.

If Culham buildings had character, so did those who worked there. Some had looked like extra's from a 1940's documentary about Radar development wandering around in beige lab coats wuth glasses stuck to their forehead, others were characters in their own right.  My supervisor, Paul Noonan, when he wasn't working on nuclear fusion, spent his time supporting the campaign for nuclear disarmament, by travelling down to Greenham common in his Hillman Avenger estate to protest the installation of US cruise missiles.  His argument was that by developing Nuclear fusion, it would remove the need for Fission reactors and remove the primary source of enriched uranium.

Also, like so many places, they depended on a steady stream of interns and students. I fell into a group who lived in Oxford and often went up there for drinks and evening out, and it is a city I still love to this day. However, mixing with them I became painfully aware of my Academic inadequacies. Most of them were sourced directly from Oxford universities, and had skills and cultural awareness that far outstripped my more humble education. However, for a while I enjoyed being in their glow as we went to the local student cinema's, and then went back to crumbling student house for vast vegetarian stews while I listened to the latest chats picked up at the latest Oxford Union debates. I am sure I was a pain to Carl, Ash, and Steve Gee, but I didn't care since I enjoyed the Bohemian lifestyle of an Oxford Student. 

Then there was Rush Common House.  One of the great advantages of working a Culham in those days was the presence of a huge boarding house which provided accommodation for people working not only at Culham, but also the nearby Harwell and Rutherford Appleton laboratories. As such was a melting pot of people of all ages who would be bussed to their destinations in the mornings. The main advantage is that as a student surviving on the wages provided, there was no need to find accommodation in a City even then expensive to rent.  It wasn't just students however, some people in their 50's and 60's seemed to be permanent residents and I often wondered about their backstory how they ended up there.

Much later in my life, I was travelling with a colleague from Manchester to Farnborough and they said they knew a great pub on the way in Abingdon. As I sat eating I had a feeling I had been there before, and I then realized that I was in the same pub just outside Rush Common House that I had been to many times before. So I asked if we could just go around the corner, only to find that Rush common House had now been torn down to become another housing estate. 

The end

With many life memories, it is hard to put them in a strict chronological order, never mind provide an exact date. However I know exactly when my Culham experience ended. I had spent the morning packing my suitcases to get the train home to my parents, then I remember going to the packed TV room to watch the start of the Live aid event on Saturday 13 July 1985. I only had enough time to take in Status Quo and Ultravox before getting my train, so missing Queen, Bowie, but getting home in time to time to cqatch McCartney play "Hey Jude". It seemed a fitting swan song.

I was 21, the placement was the completion of my Physics course, and now I had to decide what my future would be. Although I did not know it at the time, my time at Culham helped me do that.

I came away knowing that despite my Physics qualification, I did not want to work in Physics. 

at Culham, there were too many people who were so much capable and brighter than me, but were working on pitiful wages. The British capacity to undervalue science and scientist compared to say finance services was writ large there. If people with Oxford PHD's were living on such wages, what hope had I got? I was painfully aware (and don't laugh, kids) of my £200 overdraft and the need not to be a burden on my parents. However, Culham also gave me the opportunity to get confidence in my ability with computers, which channelled me into my career of the last 40 years.

My one regret is that I hoped to tell my kids that once I played a small part in solving the world energy crisis by developing clear, sustainable, safe fusion energy.  Of course that would be a total lie, but it would have been good to of at least said I walked with giants. 

When I went to Culham, commercial fusion only seemed 10 years away. Now, 40 years later, it still seems 20 years away Worse, the UK after its Brexit madness, threw away years of work with EURATOM and now will no longer be part of JET's replacement, ITER, instead deciding to go it alone and build its own commercial reactor.  One of my takeaways from Culham is that the days of British exceptionalism in large science are long gone, and we work best when in partnership. The people who worked at Culham were talented, dedicated and driven. However, they were also poorly served by a UK government that has never really understood how science and innovation happens. 

At Culham, they could at least ride the coat tails of JET. It is a pity the rest of country could no learn that we are better off when we work together. 

As to the people I worked with, Paul Noonan went onto work in super conducting magnets and cryo-geneics and Steve Gee still works at Culham. As to the rest, they are only images without surnames. 

However when I look back at my working life, despite all I have done and acheived,that brief 6 months  spent stalking those Oxfordshire halls, are still the ones that bring me the most pleasure.




 

 







Sunday 6 November 2022

We Need to Talk About Elon

 It may have escaped some people's notice that twitter is now under new ownership. The world's allegedly richest man is now $44 billion less wealthy and is now the proud owner of the world's biggest cat image sharing site

For even to a man of such wealth such as Elon, $44 billion is not to be found down the back of the sofa, so as well as personal investment, he has also taken out a large loan and brought in some trusted partners, such as that great protector of free speech, the Saudi government (Slogan - free to say anything as long as you can do it decapitated).

It is often said in the church of Musk (which sounds like a dodgy perfume name), that we are just not capable of understanding his genius.  That he moves in mysterious way, and mere mortals just don't great get his great plan.

For example, mortals when trying to decide whether to send 44 big ones would take months of careful negotiation. Not the anointed one. He toyed with the company, deviously forcing the price from 23 billion to 44 billion. Genius.

Of course, once you have bought your prize, lesser people, would have a brief honeymoon, get their feet under the table, work out a strategy, build bridges to the workforce. Not this deity. Pretty well, as soon as he took over, he took revenge over those who he felt had doubted him. I mean, what is god like powers if you cannot do a bit of smiting now and again. Then he announced charges to the blue tick system, to ensure only the true believers who have tithed sufficiently would be allowed the mark of Musk. Then he set about gutting the company, laying waste huge swathes of the work force, much like a biblical plague. 

The idea seems to be that those who are left, would be under the tutelage of disciples from his previous company and so grateful for being spared, that they will be happy to accept any conditions imposed in order for the new temple to be built.

 As a software developer myself, the bit I really enjoyed was him bringing in people from Tesla to look over the Twitter code base. That is a bit like me reviewing the Kalevala in its original Finnish. All software is not created equal, and the idea that someone could just be bought in and review some code totally outside their domain experience is both laughable and sad, and i really feel for the Twitter engineers who have to explain why it was written in Scala and not C++.

 It is also a dick move. Nothing endears you to a new boss than on his first day, him saying that up to know all the work you had done is useless. 

 To add to that there are rumours that the decision on who to let go was based on lines of code created, and in future they would use a system of stack ranking to measure performance. If you do not know that particular hell, it is a system where the lowest 10% employees are let go every month in an attempt to encourage greater efforts. Stack ranking has been blamed for the demise of GE (who invented it) and Microsoft's past failings, as it causes teams and individuals to stab each other in the back. It is almost like Elon has read a book on the world's worst software management practices, but thought it was an instruction manual.

In a way I feel worse for those who are left. I have worked in toxic work cultures, and it is not one I would inflict on anyone. Nor is it sustainable. Maybe such things work at startups, because the carrot of future IPO's are dangled in front of you to justify unsocial hours, toxic work culture etc. However,  twitter is not that, and I cannot see the present team living under the new environment, and if I was still there I would be working out my escape route as soon as possible.  Of course, that could be the big plan. Force out the high earners and replace them with interns who are both cheaper and more malleable. Still a big dick move

Of course, Elon would claim that this just a big part of his ephemeral master plan. Burn me as a heretic, but I have some doubts...

First problem is that Elon's track record is as a disruptor (Insert Star Trek Joke...Ed). His successful businesses in the past have been where he has gone against the status quo. You cannot argue that he has been highly successful in this rewriting the rules on electric vehicles and spaceflight through his force of personality. However, Twitter is not like that. It is established part of people's lives. I don't think users want disruption or change, what they want is slightly better service. It is like an old pair of trousers. Not great, but comfortable. Maybe Musk has some kind of master plan where he wants to take it (Maybe a bot that tweets for you :) ) ,but at present it feels like a rich man's toy which he was forced to buy after a late night spliff, and is now trying to make the best of it, without admitting his stupidity.

Secondly, the problem is that Twitter is not an engineering company. There is no product, and minimal actual physical infrastructure. So what do you get for $44 bill these days? Well you get a brand, known around the world, platform used by millions and a workforce with the IP capital on how to best run it, well (used to have anyway)

However, unlike rocket sheds and car factories, brands are ephemeral things. One of the mysteries of the internet-verse is why such companies are valued so highly. We have seen in the past with MySpace and Tumblr, that a brand can quickly be destroyed by alienating users.  As a Twitter user, I owe the platform nothing. Yes, I have 13 years of tweets, and a few followers, but they have little value to me or my descendents. They were just a some brain farts I had that day. 

This is the problem Elon will have monetizing the platform. As soon as you become a paying customer, your relationship fundamentally changes and to be honest, of all the social media I partake in, Twitter is the one I care least about. And that's the rub. Twitter needs us more than we need it, because its whole business model is based on attracting advertisers to partake. If the platform is deemed as tarnished, people may give up, meaning less advertising dollars. It then starts to go into a death spiral. Getting a few desperate Elonites to pay $8 for their blue tick is not going to fill that hole.

The other problem that the cost and effort to grow a rival is quite small. All you need is investment, a few server farms and a workforce with expertise in running it. Strangely enough, Elon has provided the fuel for all these and as nature abhors a vacuum, business will always will fill an opportunity. 

I do hope Elon made sure that all those employees and executives he cast off (in contravention of Californian employment law) sign a non-compete clauses, otherwise I would not be surprised to see a rival platform come along. Of course, the irony is that starting from scratch is often easier than maintaining an old system since you can incorporate all those lessons learned.(and I have copyrighted MuskRat as the new name, but I am willing to sell it for a small fee)

Finally, we have the free speech problem.

Many people congratulating Elon on his take-over seem to be the far right who are tired of group think on Parler and are relishing expressing their extremist views to an audience who have cruelly shielded from racist, climate change denying, anti-woke agenda by the Twitter illuminati. 

Problem is, not only that most people just don't to want to wade through that crap, the self-styled chief twait is now the focal point of these views. Whether he believes in them or not, he is now responsible for letting them in. His big plan is a star chamber of opposing views that will adjudicate such decisions. I can think of only one person in the world who thinks in this fractured world that would ever work, however as a suggestion as to monetize Twitter he could try selling Netflix, the live-streaming rights of the debate chamber, with added mud and jello. It could be a rating winner.

Libertarian free speech as praticed by Musk is not what most people consider acceptable bgehaviour. It is a less polite debate and more of two Rottweilers going for it until one rips off the testicles of the other.  That is what free speech without moderation is, and to be honest is really exhausting to the point where you really just want it to go away. It maybe a coincidence, but I've seen a lot more of this on my timeline recently, and the temptation to just walk away has never been stronger. 

Of course, one argument is that you should stay and add balance. This reminds me of a comedian who was invited on I'm a celebrity. At first, they thought their presence would allow them rise above the other shit eaters there, but they realised that just by being there, they also would be just another shit eater, however high-minded their ambitions were. The question therefore is really how much shit we are prepared to accept, before we say enough is enough.

Should I stay or should I go now? Should I stay or should I go now?
If I go, there will be trouble And if I stay it will be double So come on and let me know

The Clash


So what is my future with Twitter. Should I stay, and if I do where?

When I think of social media, it reminds me of gods as designed by Pratchett and Gaiman. In their worlds, gods do not come first, but are emergent properties of the beleivers. The more people believe, the stronger they become, and therefore more people believe, becoming a continual feedback loop. However, then eventually they go out of fashion, and they disappear, no longer maintained by the faith. 

Twitter is very much like that, and the question is whether I want to keep feeding this god, or should I switch to others. Of course the problem is which one?

At present Mastodon seems to be the obvious alternative, but as with most open source systems it is neither as easy to use as Twitter and also a bit rough around the edge. It is also fragmented with different servers representing different constituencies. 

This has both plus and minuses.

One of the biggest issue about Twitter is the timeline gets conflated. I follow someone because I like their photos and next minute I am subjected to their far right views because they follow Farage or another low life. At the moment Mastodon does not suffer this but it is early days. It is however more effort since I have to join multiple servers, but I must admit I am starting to like it.

Will it be as big a twitter? I don't know but it is always worth reminding people that you are not chained to site and there are alternatives

Of course I would miss some of the people I follow on Twitter if I move, but I must remind myself Twitter friendships are no more real than Elon's empathy for human beings

As a final word I know Elon is a big fan of Asimov (though strangely has failed to understand the man's warning on AI). However I would suggest a little Pratchett may teach him more more about living with humanity. 

For some reason thsi quote came to mind

There is a very interesting debate raging at the moment on the nature of sin. for example." [answered Mightily Oats.]

"And what do they think? Against it, are they?"
 
"It's not as simple as that. It's not a black and white issue. There are so many shades of gray."
 
"Nope."
 
"Pardon?"
 
"There's no grays, only white that's got grubby. I'm surprised you don't know that. 
 
And sin, young man, is when you treat people like things. Including yourself. That's what sin is."
 
Terry Pratcheet Carpe Jugulum,

If you want to join me on the other side you can find me at


@hammarbytp@photog.social
@hammarbytp@fosstodon.org 
@hammarbytp@mstdn.social
 

Wednesday 11 March 2020

Argon 1 Review



Recently I setup my own NAS server using a Raspberry Pi 4. So far it has been working nicely, however one thing that has been annoying me is the case.

When I chose the pi 4 (USB 3.0, better performance, etc) I was worried that it would suffer heat issues, since it ran a lot hotter than the Pi 3. Therefore i decided to buy a case which could include a fan. However at the time, Pi4 cases were not common, and the case I chose, although cheap, was not ideal.

The case had a fan which could be included in the case and ran continually. Unfortunately only after a few month the fan now creates an annoying rattle, which the more i hear it, the more it becomes annoying, to the point I have had to disconnect the fan.

Although it has turned out the Pi4 in this application runs less hot than I was expecting, I still wanted some head room, in case I added extra functions which increased performance requirements. Therefore I decided to look for another case. This is where I came across the Argon 1 review. Compared to most Pi cases, this looked robust and neat. Best of all it provided a power button and fan control out of the box, plus a heat sink for both the CPU and memory.

Yes it was a tad expensive, but I decided to go for it. I actually struggled to source one, and had to get it from Germany in the end.

The review


The box itself is actually quite small. The upper part is metal, and feels very sturdy, while the bottom is black plastic. It comes with vert clear documentation on installation and configuration.

The clever part is a small daughter board that plugs into the audio jack and 2 HMI ports and brings out to the front of the case. The USB-C is also brought out, meaning all the interfaces are at the front of the case in a neat package. The Pi itself is inserted into a GPIO pin female socket which presumably connects to the external USB-C slot. A fan is included in the top of the case, put is only turned on when a certain temperature is reached. Initial installation was quite quick, the trickiest part was lining up the  GPIO pins, and it requires a bit of a shove to get it all in. It was only when I had put the Pi, I discovered the cases neatest trick.

Neat Easter Egg


 A panel can be removed from the top of the case, held by two magnetic catches that provide total access to the GPIO pins, which is really neat touch.

Problems, Problems


However it wasn't total plain sailing. It was only when I re-installed the Pi on the NAS server I discovered a a number of issues. My 1st problem was installing the SD card. The case provides a slot in the back, but you can only install the card once the bottom of the case was on. However there is enough of a gap between the SD slot and the case for the card to slip into the case, requiring you to remove the bottom to access it

My 2nd problem was when i re-installed it on the NAS, it would not turn on.  Actually it was hard to know what it was doing, because I could not work out whether I could see the Pi status LED's or not.

In the end it appeared that the case has slightly higher power requirements that the standard Pi, and since it was being powered via a USB hub, clearly there was not enough. Once the Pi got enough power, I could see you could see the LED's through the translucent cover, but I would of preferred if  they had been brought out of the case via light pipes.

However I would recommend that the case bottom is not screwed in until, you are sure everything is working.

Once sorted, things went smoothly and I am very pleased by both the look and function of the case. If I was looking at creating a General purpose Pi computer, I would definitely choose this box.






Thursday 2 January 2020

Build your own NAS server



One of the banes of modern technology is how to keep your digital information safe.  While modern PC's are quite reliable, often you are only one hardware failure away from losing irreplaceable documents, photos and videos. Therefore it is critical that you have some sort of backup solution.

The solution  will often depend on the size of the files. Documents can often be backed up to the cloud using things like box or google drive, but cloud storage has its own issues and I would nor rely on it. While for small amounts of data it can be cheap and convenient, even with super-fast broadband backing up even moderately large files can be tedious and soon you will find yourself paying yearly subscription fees as your storage requirements increase.

The alternative then is some sort of local backup. I take a lot of photographs, and recently I have been continually hitting the limits of my disk space, which has forced me to continually prune images. It seemed like a good time therefore to look at some sort of storage expansion.

My requirements we as follows

  • Easily accessible - accessible through all our computers and via internet
  • Data Redundancy - loss of one component would not result in loss of data
  • Expandable -i.e. capable of adding extra storage when required
  • Extendable - Add extra features if required
  • Low Power - does not use much energy
  • Cheap - not expensive :) 
Over the year I had tried a number of backup solution. I started with recordable DVD's until I found the shelf life of DVD's was only 2 or 3 years and my data requirements quickly outstripped the amount data I could store. I then moved to flash disks, but that is quite an expensive per MB way of doing it. My current solution is to back up to standalone hard disks. While this is relativity cost effective, they are not very accessible and for that reason backup rarely occur more than evey 6 months, meaning that I am living on a tightrope for the rest

The obvious solution is a NAS box (or network addressed storage). This is a box that can be connected to a Ethernet cable that allows you to transfer data as if it is a hard disk. NAS boxes allow you to have 2, 4 or even more disks in a redundant configuration.

You can get a two bay NAS box for about £130. However you then need to populate it with hard disks, which means, say a 4GB 2 bay NAS box would be about £330. Not terrible, but I felt I could do better

Build your own 

The alternative is to build your own. First possibility is to re-purpose some old PC. Even a relative low power PC with two SATA interfaces will make a perfectly good NAS box. However it will be quite power hungry and noisy

The other option is to use something like the Raspberry Pi. These arevery low power, virtually silent and have good software support. However until recently, the Pi was not a good NAS server device. Firstly Pi's do not have native disk drive interfaces requiring you to connect via USB. However Pi's only supported USB2, which was relatively slow. Worse, it shard the USB bus with the Ethernet, meaning that if you are sending data and writing to disk via USB, it would cause blockages.

However recently the Pi 4 came out. This has a separate Ethernet/USB channel. Even better it supports USB 3. the difference is huge with a 10 fold increase in Ethernet throughput and a 4 fold increase in USB throughput. The Pi 4 now has a huge potential as a NAS server 

A 1 Gb Pi 4 is about £30, so is good value. All we need to do is work out how to add some disk drives and add some software

Adding the disk drives

Without a native SATA interface, you are reliant on USB to connect to a drive. One solution is a USB to SATA interface. One of problems with this however is that you will need a separate power supply to power the disk. I wanted a few power supplies as possible, so I wanted to stay away from this. 

The alternative  is a hard disk in a USB enclosure. A 4TB USB 3.0 portable disk can be powered via USB 3.0 and strangely the price is almost exactly the same as the native disk. One downside is that the disks are likely to be slightly slower, but for the work they were to do, this was not likely to be a major issue.

Another issue is whether a Raspberry Pi would have enough power to drive two USB 3.0 hard drives. It looked marginal, so i added a powered USB 3.0 hub. The hub I chose not only powered the dives but could also power the raspberry Pi and would allow me to add more drives if required in the future. The only downside was that it was a bit larger than I wanted.  

So finally I had my bits together as shown below, saving about £70 over the NAS solution.

Parts list

Raspberry Pi 4 1GB - £35
Fan £10
Pi Case £5
SD Card £10
2 WD USB 3.0 4TB disks £85
Atolla USB 3.0 Hub Powered, 11-Port USB 3.0 Hub £30

Total price £260

(The case and fan may not be needed. The PI 4 runs hotter than the Pi 3, but in this configuration it is not doing much. The fan was just a precaution, while the case was just to tidy things up. )

Getting it working

The next issue was getting it working. Here I had two solutions. Firstly set up the OS myself or install a pre-built package. I just was not in the mood to do the first, so it looked what free packages were available.  It basically came down to either FreeNAS or Open Media Vault(OMV). Both would of been fine, but OMV seemed a little bit more Pi friendly and had lower hardware requirements.

The first thing was to set it up. OMV does not have a complete raspberry Pi distribution. Instead you install the Pi basic distribution and install OMV on top of it. The complete instructions are here

Installation went smoothly,, so the we moved onto the OMV setup guide.

First decision was how to format my drives. I they were to be windows drives we would use NTFS. However linux NTFS support is not as efficient as windows, and because the drives would never be attached directly to the PC, it made sense to use a native Linux format. therefore we formatted them as EXT4 drives.

It was then we hit an issue.

RAID

The original idea was to have my two disks in a RAID configuration. So what is RAID. RAID allows you to group multiple disks in such a way that they look like one logical unit. There are about 6 different ways you can do this, but in practice most people choose 1 of two.

RAID 0 makes multiple disks look like one large disk.
RAID 1 combines 2 disks in such a way that the data written is duplicated or mirrored across both disks

However deep in the setup guide we get this.

RAID+USB= Potential Problems 

Setting up RAID of any type using “USB to drive” connections is discouraged. 

RAID over USB has known problems. The USB interface (there are several flavors) may filter some the characteristics of the drives, fail to pass SMART stat's and ATA drive commands, delay the assembly of a RAID array, etc. While USB may work in some RAID cases, it's not as reliable as using a standard hard drive interface. If RAID of any type is considered to be a requirement, drives should be connected with SATA or SAS ports. If users choose to use RAID over USB connections, it is done at their own risk with the potential for the total loss of stored data. RAID issues involving ARM boards, USB connected hard drives, or USB RAID enclosures are not supported on the forum. 

Basically raid over USB was not fully supported.  I think if I was running OMV this is information that I would put front and center and not hide it mid way through a setup document. Oh, well time for plan B

In these situations it is worth going back to first principles and see if the path you are on is where you want to be.

Clearly I wanted redundancy, but did i need RAID to do that? I chose RAID because that is what most enterprise systems use. However they have multiple users writing loads of files to disk frequently. My box on the other hand will have a few users writing infrequently to disk. In that situation RAID is not so useful and actually adds a unnecessary overhead.

Solution


So whats the alternative. Basically I want to copy files changed on one disk to another one. Linux provides a protocol to do that called RSYNC. It basically checks to see if any files have changed and copies the files to another destination. Fortunately RSYNC is supported by OMV, so I can  ask the Pi to check the main disk every set period and copy changed files to the backup disk. Here is information on how to do that 

This would not be as immediate as RAID and there is  small window where changes may not be copied, but the likelihood of a disk failing during a RSYNC process is very small, and far lower than an issue with RAID affecting both disks.

So with RSYNC setup, I could attach to my PI via a Samba network connection. On my windows PC it looks like a new drive and i can just drop files onto it. Every 12 hours, changes to the main disk are copied to the backup disk.

So after a test and 1TB of data transfer later I have working NAS server!

So did I meet my requirements?

  • Easily accessible -SMB shares mean all computers on the local LAN can access it. Internet access is still to be done. 
  • Data Redundancy - 2 hard drives should provide considerable redundancy
  • Expandable - USB hub allows extra drives to be added
  • Extendable - OMV plugins provide extra feature support
  • Low Power - PI is very low energy
  • Cheap - I think it is about as cheap as I can make it. 
So is building your own worth it?

In terms of effort it was relatively easy, however there were some cons.


  • Lack of decent housing - there just does not seem to be any 2 slot boxes out there of decent size of a Pi. Weird
  • Cost difference not that great - I was hoping for a 50% savings, but in fact the hard drives were the biggest cost. Considering there was no hot swap drives or metal cases involved the cost saving were more modest than I hoped
  • No RAID support - if I knew USB RAID was not supported when I started then I might not of gone down this route. However in hindsight RAID is not as important as I thought
On the plus side 

  • It was cheaper
  • It is more extendable
  • It was more fun

Work to done

Firstly was it worth it? Some may consider thee £70 saving over a regular NAS to be relatively modest, however that ignores that I can easily extend it by adding more or larger disks.

There are also other things I want to do. First I have not tried wireless transfer yet. The ability to add or retrieve via wireless devices would be useful. Secondly I would like to allow internet access. This will be a little more tricky, but the ability to access photos from anywhere would be great. There maybe other things to add in the future as I explore OMV more

Finally, at the moment it consists of 4 devices connected via Velcro. I need a better housing. There really is a market out there for a pi case with two disk slots, because I could not find one. While 3D printing could be an option,   at the moment I am considering  a Tupperware container, wooden box or my old standby , a cardboard box.

Performance



Tuesday 19 November 2019

Coloring Old images



Taking an old B&W  image and coloring it has become a thing in recent years. When done well it can add life to a image and bring new insights. However generally it takes a lot of work in photoshop and a trained technician. However even done well, it can have issues because often the colorer has to guess on the color of clothes etc

I was interested to learn therefore of work based on machine learning to do the same thing. The idea is to train a algorithm based on a set of images and then use the result to color unrelated images.

I was skeptical initially, but the results looked good so I thought I would have a closer look. I wanted to see if I could run the process myself. The code can be found here

https://github.com/jantic/DeOldify

and instructions are here

https://github.com/jantic/DeOldify/blob/master/README.md

 However it looked like a lot of effort to set it up, but fortunately there was an easier way

If you go to this link

https://colab.research.google.com/github/jantic/DeOldify/blob/master/ImageColorizerColab.ipynb

You can run a virtual machine totally in your browser. If you click it you will see  a page like this

All you need to do is click the boxes in the right hand column. Each time it will do some part of the setup is carried out. Wait till is complete then do the next one

Finally you will reach this section



This is where you do your work. First you need to provide an image.

At this point I have not found a way to load an image from disk so you need to upload your images to the internet. An important fact however is that the URL of the image must end in .jpg. I use imgur.com as my repository which is free. Upload your image and copy the URL into the source url box and then hit the start button under Colorize. After a few minutes it should present you with a screen containing a full resoloution coloured image, a smaller version and the original B&W

I have not found a way of saving the image directly, so instead right click on the full res version and hit save as. You can then save it as a .png file

If you have problems, try reducing the size of the original image.

The results


The results vary depending on the source of the original.  The better the scan, generally the better the image, however the compare well with the hand created version. Plus its free

You can also play with the render factor controls.