Tuesday 31 July 2012

Day 7 - Can Team GB finally win a gold?

digiguide.tv is missing channel 24 yet again (I've posted on their forums to notify them of the problem - almost all their listings to 10th Aug have nothing on channel 24!), so my initial figures of 23 channels, 23 sports, 56 slots and 212 hours and 38 mins of recordings may need revising if the Now/Next data pops up a sport on channel 24 later today. Yes, I'll be splitting the usual suspects (tennis, sailing, equestrian), so I won't bother mentioning the splits in the future unless a new sport is involved that I haven't split before.

You may have noticed that I'm using day numbering three ahead of what the BBC are saying. The reason for this is that Olympics events did not start on Saturday (that's the first full day of events admittedly), but actually three days earlier. Yes, there were "only" 6 women's football matches, 8 men's football matches and a scandalously unaired archery competition on those three days, but they were all part of the Olympics and should be counted as taking place on an Olympics day.

So we now reach the end of a week's events and Team GB are desperate for a gold medal. Just don't ask me where the first one will come from, but if they don't get one today, I think I'd call the first week a failure. A 15-year-old girl won a gold medal in swimming for Lithuania yesterday and even they're beating us in the medal table now.

Watching bits of the artistic gymnastics yesterday - well done for the bronze medal, lads! - it did confrm what I was thinking before the Olympics was needed to cover the event: multiple channels surely? There were clearly multiple apparatus in use by competitors simultaneously, so we actually missed quite a bit of live action (some of it was shown recorded, but I suspect we didn't get all of it even then). Athletics will be in a similar position of course.

Just finished watching an epicly long tennis match between Tsonga and Raonic on BBC Olympics 7 HD today. A real shame the terrestrial BBC channels didn't pick up on it - no British player in it I guess. It ended 6-3 3-6 25-23 to Tsonga! That's 66 games and something like 3 hours and 57 minutes. I believe the number of games is a record for an Olympics tennis match, though the points were almost all short and there were no tiebreaks, so I'm not sure that the total time sets a record. No commentary on most of the match and I actually quite liked that - plus the umpire was announcing things with quite a sarky voice :-) At 21-21, a Scottish commentator finally turned up and then Tracy Austin joined in at 22-21.

In case you're wondering what happens when the first of my eight 3TB HDD's gets close to full, I'll explain the setup to you. My transmedia program will only copy to an HDD with at least 1% free (something like 29-30GB) and also with enough actual free filestore space to copy the file into. The disks to be checked are listed on the command line (e.g. /mnt/disk1 /mnt/disk2 /mnt/disk3) and each one is tried in the order specified. If none have the space, the file remains on SSD.

So what happens when all the 3TB drives are almost full? I have 2 in one PC and 3 in another, so when a full day would fill the final disk, I simply swap out any one of the nearly-full 3TB disks for a fresh empty one before the first recordings of the day start. I have three 3TB in a drawer and another two 2TB as emergency spares. I would also remount the "full" disks read only to avoid anything being written to them. Clearly, there must be at least two 3TB disks in each PC at all times for all of this to work. Note that I use /dev/sd* syntax in /etc/fstab and not the UUID (which would change each time a disk was replaced).

I picked up one trick in the last few days with my setup - after the final slot to start is allocated to a tuner in the evening (it's currently boxing at around 20:25 or so), then I double up as much recording as I can on both PCs. The reason I don't do it earlier is that it's possible to over-allocate double-ups and lock out recording later slots. Those later slots could need a 5th transponder that's not available because 4 tuners are using the first 4 transponders already. There's no SSD space issues because there are far less evening slots than afternoon ones, at least in this first week.

Well, the answer to this particular posting title was a sad "no". A week of events, something like 55 golds awarded and we're now probably at the lowest position (21st) in the medal table that we'll be in for the entire Olympics. The BBC Website (I'll eventually find the link again!) showed each day's progress in 2008 compared to 2012 and we are already behind that (we won a gold on full day 2 in 2008).

 Day 7 recording size: 1011GB - grand total so far: 5088GB (5.0TB)

Monday 30 July 2012

Day 6 - Still a packed schedule

Today is only a fractionally less busy day than yesterday with all 24 channels used again, 23 sports involved and 58 recording slots totalling 220 hours and 40 minutes. Apart from digiguide.tv missing channel 24's events completely, the other interesting note is that the BBC has finally decided to experiment with splitting up some of the tennis slots into two, which is something I've been banging on about for days now.

Channels 22 and 24 will broadcast tennis in 11:30-16:00 and 16:00 to 20:10 slots, which is great, but sadly the mad 8+ hour tennis slots remain on channels 6 and 7 (I will be splitting those along with sailing, which is still over 6 hours today). The BBC have also finally mostly split each football match onto separate channels, though channel 4 wrecks it with a 2-match 5 hours 5 mins slot at 14:20. So better scheduling from the BBC at last, but by no means perfect.

It's slightly worrying that Team GB have no gold medal yet and languish 16th in the medals table after 5 days of events. We were far too optimistic about Cavendish for road racing gold the other day, IMHO. If anyone's seen the Tour De France over the years, they'll know it's an unpredictable sport, particularly if a strong breakaway occurs.

Also, I was surprised how all the British riders behaved like they were a team and didn't have anyone try to join the breakaways - it was an individual event after all (there are no team prizes or glory like in the TDF). As an individual rider, Cav's complaints about other teams not working shows the wrong attitude - it was every man for himself (the winner wasn't working with anyone!) and this was the flaw in all the British riders' tactics.

A couple of times today, the Now/Next info has actually managed to double up recordings in tvheadend for the same satellite channel! Something seems to pick up an empty slot title (e.g. "This is BBC Olympics 9 HD") and put it into the current active recording slot and, somewhat bizarrely, also records it despite it not matching any of my Automatic Recording strings (I deliberately made sure that empty slot title didn't match, otherwise I would end up with thousands of  hours of timetable video :-) Luckily, the original recording in that slot continues as well and I just leave both running unless there's SSD space issues.

I just realised I didn't show some of the output from the script I wrote the other day to keep an eye on the recordings from a filestore point of view. It was mainly written to make sure that what was on the tvheadend Web interface matches what's on the filestore. So here's an example from today (I have the script on both pc1 and pc2):

  4992 MB: BBC-Olympics-2-HD/-Olympic-Gymnastics-Artistic-.mkv
18320 MB: BBC-Olympics-4-HD/-Olympic-Tennis--1.mkv
11843 MB: BBC-Olympics-5-HD/-Olympic-Tennis--1.mkv
 9616 MB: BBC-Olympics-6-HD/-Olympic-Sailing--1.mkv
21932 MB: BBC-Olympics-8-HD/-Olympic-Hockey-.mkv
 8242 MB: BBC-Olympics-12-HD/-Olympic-Weightlifting-.mkv
 4877 MB: BBC-Olympics-13-HD/-Olympic-Basketball-.mkv
16437 MB: BBC-Olympics-14-HD/-Olympic-Beach-Volleyball-.mkv
 9908 MB: BBC-Olympics-15-HD/-Olympic-Archery-.mkv
 6137 MB: BBC-Olympics-20-HD/-Olympic-Table-Tennis-.mkv
 7327 MB: BBC-Olympics-21-HD/-Olympic-Tennis--1.mkv
13610 MB: BBC-Olympics-23-HD/-Olympic-Boxing-.mkv
Active: 12, total: 133247 MB (130.1 GB), free: 32050 MB (31.3 GB) = 16.32%
16675 MB: BBC-Olympics-19-HD/-Olympic-Badminton-.mkv
Done: 1, total: 16675 MB (16.3 GB), free: 32050 MB (31.3 GB) = 16.32%

Any filename with "--1" in it is a second file that is recording after a split (i.e. I split Tennis and Sailing today). The first file will already have been moved to HDD of course. You can see a 16GB Badminton file that would be moved to HDD a few minutes later by my transmedia program, which freed up that space on the SSD. If anything appears in the Done list that shouldn't, then I'll know something is wrong.

One handy thing to do is to keep a rolling set of 2 runs of the script output in the terminal permanently, so you can compare between runs and see what's changed. I tend to run the script when multiple recordings start simultaneously (which unfortunately happen a lot!), to make sure things look OK.

On another note, I've been running a BBC One HD feed in XBMC permanently during the day (switching it occasionally to a sat feed when the BBC One HD sport is boring). Prior to today, I'd only put it on occasionally if there was really something important to watch.

A little sad to see that after 6 days of events and the awarding of 38 gold medals, Great Britain has failed to win a gold and has slipped even further to 20th in the medal table. Let's hope that we climb up the table soon! An impressive start by France I see - thanks to their swimmers, they've got 3 golds and lie in third place in the table.

  Day 6 recording size: 1104GB - grand total so far: 4077GB (4.0TB)

Sunday 29 July 2012

Day 5 - Busiest day in the first week?

Despite being a Sunday, it's not a day of rest for Olympians as we have record numbers of sports (24), satellite channels (24), satellite recording slots (58) and recording time (226 hours) to deal with today. To cope with this, I'll be splitting equestrianism (split at 13:25), sailing (15:00) and tennis (15:45 or 16:05) into two recordings because they're all crazily over 6 hours each. I may do each split a little earlier if 90% SSD usage on pc2 is reached before then and if things get really tight, I'll look at splitting the football at 14:30 too (5 hours 20 mins for two fixed-length matches that last about 2 hours each! It's totally criminal the BBC don't have those in 2 slots with at least an hour gap between the slots).

Other PC adjustments include suspending at 1.15am, 1.30am or 1.45am (whenever the machine is idle and has no-one logged into it) with a wakeup 15 mins before the first event in the morning (though I'll power it up an hour before the first event anyway to do my system checks). On both pc1 and pc2, I've removed transmedia's verify pass and also reduced the check interval for files to copy from SSD->HDD from 5 minutes to 3 minutes (two successive checks have to have the same timestamp and filesize for the file to be copied).

Each day, I check all the tuners are reporting 100% quality and I also check whether duff tuners 2 and 4 on pc1 are working. This morning saw a bit of life from tuner 2 - the transponders reported OK for the first time since the fault, so I've decided to try and use tuner 2 today (handball is recording on it as I speak). Tuner 2 is now also recording the dreaded 7 hour equestrian slot.

Now tuner 2 in pc1 is operational, I've removed the recording of Freeview 304 (HD version of 301) again and this has balanced the SSD usage between pc1 and pc2. The extra tuner available in pc2 (used for recording 4 HD channels) is counteracted by recording 2 HD and 3 SD terrestrial channels on pc1.

One thing I can look at is possibly doubling up now on one or two recordings. I'm experimenting with the 12:00 tennis session on both pc1 and pc2 since I'll be splitting both at 16:05 and can do the split a minute apart to avoid losing any video. Yes, it did occur to me that you might be able to record the same slot on the same PC and tuner twice, but tvheadend won't let you do that (presumably once the process has the tuner stream for the initial recording, it can't access that stream a second time).

So far so good with balancing the SSD space and tuners today, but it struck me that after 14:40, there's only a couple of recordings finishing until 16:25 (so only a little SSD space can be freed up except for my splits of course), but loads of stuff - 9 recordings! - fires up between 14:20-14:55 that accelerates the SSD usage from as low as 30% right up to 80% or beyond an hour or two later (I'm hoping my splits will keep it below 80%). I suspect we'll see this pattern throughout the Olympics, because it certainly happened yesterday as well.

The good news is that I got through the peak recording period with a maximum of 83% of the SSD space used on either PC. There are less simultaneous sessions in the evening than in the afternoon, so there's no issue with space for the rest of the day. I still won't be able to watch my recording of today's F1 GP until about 20:30 tonight, because all evening sessions will have started by then and they can be run unattended.

Another day over and I've written a script to show me the active recordings and the space they use on the SSD (and the free space left on the SSD), plus any that have finished and not yet copied to HDD. It's just for reassurance that the "recording blobs" in tvendend match up with what's going on w.r.t. the filestore.

 Day 5 recording size: 1192GB (a new record) - grand total so far: 2973GB (2.9TB)

Saturday 28 July 2012

Day 4 - Now we hit the big time

Today is the first time that my recording system will be truly stress tested, with a packed schedule of 51 recordings across all 24 satellite channels (plus all the terrestrial recordings on top of that). This means it marks the debut of BBC Olympics HD channels 4-24 in showing actual live Olympics sport (rather than timetable video or the torch relay).

My quad sat tuner cards are actually two twin tuners on one circuit board, so with tuner 4 on one card out of action, it was predictable that another tuner would fail shortly after and that's what happened this morning (tuner 2 on pc1 - I guess they alternate on the connectors?). The good news is that I mis-typed one of the transponder numbers in the program I finished yesterday, so it does look like you can just about scrape by on 6 tuners today with absolutely no overlap though.

What I've done is allocate all sports strings to Automatic Recording on both PCs and then manually abort events which one of the two PCs can't find a tuner to record on. I can now see five 8+ hour tennis sessions today starting at 11:30 or 12:00, so that's going to be "fun". So far, I've managed to record everything, but it's probably my first all-day "dressing gown" session of the Olympics :-)

As I suspected, it got very tight at lunchtime on pc2's SSD since that's the more heavily loaded of the two. I ended up stopping and deleting the terrestrial recordings on pc2 because I'm doubling up on all of those. The problem is that the BBC haven't split up long slots into sub-events (tennis being the worst - 5 slots of over 8 hours each) so many of them start in the morning and either break at lunch or go on right through lunch continuously, so 4-6 hour single slots are quite common.

This means my SSD->HDD copier has more simultaneous long slots to deal with and hence a lot more SSD space is needed. As a precaution, I've now turned off all terrestrial recordings on pc2 and will just record them on pc1 only from now on. Things have calmed down since I've done that, with pc2 SSD usage around 50-60% now (but this will climb this afternoon of course). The only actual gap between events starting today that is longer than an hour is at 15:25-16:50, which I finally get a break (once events are recording, I can relax a bit and leave them unattended).

The latest starting time today is 20:25 (boxing), so once that's started recording, I'll get to watch the recording of today's Hungarian F1 GP qualifying which has been rudely shoved to BBC Two and in SD only :-( At least the race tomorrow is on BBC One HD - yay! - although only a 90-minute long highlights show sadly. The only other non-Olympics activity tonight is the lottery show at 10.20pm on BBC One HD, where I'll be updating my unofficial lottery site at the same time as the show airs (unlike Camelot's official site...).

Around about 15:30 today, pc2's SSD started to exceed 90%, so I found a way to free up space manually. I stopped my transmedia program temporarily so I could move stuff without a verify (and 5-10 mins earlier), then I targeted the ludicrous 8+ hour tennis slots (8 hours and 40 mins is the longest - 2 hours longer than today's road race slot!) and split them at the half-way point like the bleeding BBC should have done. I did this by stopping the recording for a few seconds and starting it again - the fresh recording uses a numerically incremented separate filename and then I could move the original (4+ hours!) recording from SSD to HDD and free up some 20-25GB of space in one go. I will probably have to do this wiith anything that exceeds a 6 hour slot in the future.

I didn't know what to do with Freeview channel 304 (an HD version of red button channel 301) when I was first setting up the terrestrial channels and I decided that because I was recording all the sports in HD already and also 301 in SD, it would be a duplicate almost all the time. However, now I'm down to recording everything just once, I've activated the recording of 304 as a precaution on pc1 (which is fine with its SSD free space at peak times).

I guess hindsight is a wonderful thing - if I'd have known about all these problems in advance, I'd have got dishes with octo LNBs instead of quad LNBs, four quad tuner sat cards instead of two and 480GB SSDs instead of 240GB. That way, any two tuner failures wouldn't have mattered and I wouldn't nearly have run out of SSD space either. Plus I'd have been able to record everything twice, which would have been a bonus and would allow me to leave the whole setup mostly unattended.

I made the mistake of believing the blurb for the sat cards when they said they could record 48 HD channels simultaneously. The problem with the Astra satellites is that their transponder bandwidth is around 45Mbit/s and the BBC uses around 10-11 Mbit/sec for their HD channels, so there was never going to be enough tuners (4 tuners * 4 HD channels = 16) with a single quad card to cover the 24 channels. The cards took weeks to arrive and I finished my sat installation a few months ago, so I'd already lost some redundancy before the 2 tuners gave up on me. If it was as easy as just ordering two more cards, I'd have done it, but it would also require a quad LNB -> octo LNB replacement and another four cables off of each dish, which is yet another callout for a sat installer.

My Automatic Recording strings picked up Now/Next info for unscheduled fourth and fifth 8+ hour tennis slots (11:30-20:00) on channels 23 and 24, which I recorded (with mid-slot splits like I mentioned earlier). I say "unscheduled" because digiguide.tv's site that I paid a subscripion for failed to list anything on channels 23 and 24 today :-( It definitely pays to use the Now/Next (EIT) system I guess!

I actually went to bed early tonight (yes, before the last events had finished recording!) because I was so tired monitoring the recordings. The problem is that tomorrow is even busier, but now I know what to do with the long slot events, things should be a lot less frantic (i.e. not 90%+ full on pc2's SSD). Apart from serial offenders equestrianism (7 hours) and tennis (8+ hours), it looks like sailing (over 6 hours) is another sport for the split treatment for overly long slots.

Day 4 recording size: 961GB (a new record) - grand total so far: 1781GB (1.7TB)
In case you think I'll run out of space in a few days, don't worry - I have about 25TB of HDD space available for the recordings (8 * 3TB and 2 * 2TB - you never get the full 2TB/3TB remember).

Friday 27 July 2012

Day 3 - Wherefore art thou archery?

If I were an Olympic archer (especially a British one in my home Olympics), I'd be somewhat miffed that the BBC is staggeringly refusing to air any of the first day of my sport, despite having 24 (and more!) channels available doing nothing until this evening's Teletubbies opening ceremony (which I won't watch live cos the EuroMillions on BBC Two is actually more exciting).

The "Now and Next" info on the 24 BBC Olympics channels this morning has no mention of the sport, neither does digiguide.tv (either in the "normal" BBC channels or in the 24 Olympics channels) and it appears that even the BBC Website has an ominous "No video coverage" for 8 hours that the archery ranking competition is on. Yes, it's just a method of seeding who plays who in the "real" rounds, but it's still part of the Olympics and really should be aired. Shame on you BBC!

Archery update 1:
To compound the mistake the BBC made in not airing any of the 8 hours of archery today, two world records - both involving a blind man! - were set in the archery ranking competition that the BBC deemed apparently not worth airing. Yes, the whole of the UK missed the first two world records of the Olympics thanks to the BBC's ridiculous omission of any actual Olympics sport on their 30-odd channels today. What's even more galling is the world record article I've linked to now has bleeding video of the competitors in action (complete with the standard event captions etc.). Yes, the BBC cameras were actually there, filming it all and then not broadcasting any of it - arrrrgh!

Archery update 2:
Even in real life, it appears much confusion (and a fake ticketing web site) led to many people thinking they could go in person to the archery ranking competition and watch it because it was deemed "non-ticketed". However, the Guardian reports that it was always intended to be behind closed doors with no spectators. With the BBC not airing it either, this surely must rank as the least viewed event of the whole of the Olympics? A real shame, because a blind archer setting world records is a great Olympics story - sadly, all we're left with as a record is an 80 seconds video and a few paragraphs on the BBC Web site.

I just powered on pc2 this morning and tried re-configuring sat tuner 4 and - lo and behold - it's now working perfectly. This is quite mystifying because I did a load of test recordings on both PCs with up to 20 simultaneous channels without an issue (we've had 6 or 7 simultaneous so far). I have some time until the evening opening ceremony to test this out (some dummy recordings etc.) and decide whether to use tuner 4 at all. Most of today, though, will be spend finishing the coding for my "find the best Automatic Recording strings per PC" that I started coding in C yesterday.

What's slightly bizarre on the radio front though (and they ignore archery too) is that Radio 5 Live is spending a ludicrous amount of time today covering the "build-up" - the whole channel is dedicated to that from 10.00am this morning onwards, even pushing the more important Hungarian F1 GP practice sessions - which is actually, you know, live sport - to Radio 5 Live Sports Extra! At least the BBC TV channels are only airing a few build-up programmes during the day, which I'll be recording. I'm not going to record any of that radio build-up coverage though - far too much of it with no live sport (which is the whole point of that radio channel!).

It doesn't help casual viewing of the 24 Olympics channels that the BBC have three major issues with their 24 channel allocations:
  1. I was under the impression that with 24 channels, there would be an absolute maximum of two sports on a particular channel (and they wouldn't wander channel numbers between days, which they sadly do) and only then when there was an unavoidable clash due to a busy day across the board. At least half of the time you'd hope for one channel per sport. Nope, the first full day on Saturday 28th July has six channels (1, 2, 7, 8, 9 and 12) each all showing three different sports during the day on the same channel.
  2. The number of channels carrying a particular sport on a particular day should also hopefully be limited to the minimum required for actual schedule clashes of the sport. Again, half the time you'd hope for one channel and perhaps two (or even three) when there's simultaneous events in the sport. Nope, the BBC has allocated four channels (3, 9, 13 and 22) to badminton on Saturday 28th July, made even worse that channel 3, 9 and 22 badminton could have been put on one channel, because they don't overlap themselves at all!
  3. After the first two points showing how haphazard the scheduling is, the BBC compound it by leaving channels 23 and 24 completely empty on Saturday 28th July! A huge waste of half a transponder on a busy Olympics day that no-one will bother viewing or recording (it'll just be timetable video and the dreary Olympic theme).
Luckily, I just finished coding my program to use Automatic Recording strings in tvheadend ("Olympic Equestrian" and so on) with a few hours to spare. The bad news is that if you have two PCs with a quad tuner card in each like I do, the minimum tuners you can use to cover tomorrow's over-crammed schedule is 7 (3 on one and 4 on the other). Here's some of the example output for Saturday 28th July:

22 channels/19 sports/49 slots totalling 200 hour(s) 12 min(s) recording time
Earliest time: 08:22, latest time: 24:15 (this includes 2 min pre/5 min post padding)

With 3 tuners in each, allocations work only until 09:17 on Saturday, when volleyball on channel 18 wrecks everything and can't be recorded.

With 3 tuners in pc1 and 4 tuners in pc2, the allocations work, with 16 of the 19 sports on pc1 and 15 of the 19 sports on pc2 (11 of the 19 sports are doubled on both PCs). Because of multiple sports on certain channels, this does work out on pc1 (in case you think it only has 3 * 4 channels it can record from - we're talking sports, not channels here).

I have some optimisation code that tries to double up on my favoured sports if they're not already, but I suspect it won't work with only 7 tuners. I may try it at some point though. In case you're curious, it's 427 lines of C code so far including comments, but it's not had some user-friendly code added for parsing command line options yet.

Tomorrow is getting me a bit worried now - with me being one tuner short and hopefully not going to get transponder clashes with the automatic recording strings, I can see a hotspot at 09:17 tomorrow as I mentioned. I'll be nervously checking the system at that point - if it gets past that point, my code works. If not, there'll be a frantic scamble (to do what?! Stop some of the double-ups until the programme can be recorded?).

It'll also be the first stern test of my "transmedia" program which copies recorded programmes from SSD to HDD with a verify pass and a file sync to confirm it's written to physical disk. It can do 50-60 Mbytes/second, but it could get tight on free SSD space, particularly with the two 8 hour slots I can see for the tennis tomorrow. It's the long slots that are the problem - they leave 10's of GBs of recordings on the 200GB SSD that I can't copy/delete until about 10 minutes after the recording ends. I think I will have to monitor the SSD space free on Saturday evening - one trick is to stop the recording as soon as the action ends (i.e. it goes back to the timetable video), which means it'll get copied off a bit earlier. One thing I can do with transmedia is turn off the verify pass, which cuts a lot off the copy time at an increased risk of bad/failed copies not being spotted.

Opening Ceremony update 1:
Although I won't be watching it live, I've just realised I will be recording the opening ceremony fourteen times over (10 in HD, 4 in SD). WIll this be a world record for an individual recording the Olympics opening ceremony? Yes, this includes all three versions on Olympics channels 1-3 twice over and even the 3D version (on BBC HD) twice over, despite me not having a 3D set!  I believe this may well be the first BBC Olympics 3D broadcast, right at the end of day 3, sigh...

Opening Ceremony update 2:
In a predictable scheduling cock-up, the Opening Ceremony programme overran by 21 minutes - it's a good job I tuned in at 12.30am expecting to see closing credits, only for the ceremony still to be in full swing. ho hum. I quickly added some extra programmes to the schedule manually - it's always a bit of a panic when there's no EPG entries for it! I wonder how many people got knackered watching, set it to record the rest before going to bed and then missed out on Sir Paul McCartney and the finale completely? The pre-ceremony rumour was that the show was to be cut short a bit, but obviously that wasn't the case!

Day 3 recording size: 217GB - grand total so far: 820GB

Thursday 26 July 2012

Day 2 - Now it's the men's turn

I'll be spending the next few hours looking at the satellite tuner problem on the first PC as we move into the second day of events, this time with 8 men's football matches. I was a bit annoyed that the BBC Olympics HD channels have been "double heading" matches in their EPG/Now and Next slots, so there were several 5 hours 5 minutes slots yesterday with 2 matches in each slot. This is poor on two fronts - one is that you'll have to record both matches together as a single file and then video edit them later and two is that there was a big gap in the middle of the 2 matches where nothing was going on anyway! Never mind the two sets of video edits I'll be doing for that Columbia vs. North Korea farce last night...

It's an earlier start today, so broadcasts begin at 11.50am and it looks like three BBC Olympics HD channels (1-3 again) are in use today. Sadly, I'm not only any closer to finding out why sat tuner 4 on pc1 isn't working, but - unbelievably - sat tuner 4 on pc2 is exhibiting the same symptoms! For the moment, I have removed all transponders from tuner 4 on both PCs and am recording BBC One HD and BBC HD on terrestrial on both PCs, so there is no loss in recordings at all.

Yes, the double failure in an identical way made me suspect the latest drivers for the sat card, so I rolled them back a version and it made no difference. pc2's tuner 4 failure is slightly different though - there's a mix of "Constant FEC", "Bursty FEC" and one transponder even has an "Ok" status!

Anyway, enough of my woes, I did have some success with some coding in preparation for Saturday's first full day. At the moment, I've got recordings triggered by the phrases "Olympic", "Olympics Football" and "Olympics Fencing", which works when there's no sat tuner clashes on Wed-Fri, but Saturday uses 22 of the 24 channels and as I start adding more Olympics phrases, it's highly likely I won't be able double up everything on both PCs.

Hence, I'll be writing code to do the following:
  • Get the satellite EPG for the 24 Olympics HD channels and trim it down to a simple data file per day with a line for each EPG slot (this has been done): channel_number|start_time|end_time|Olympic <sport name>
  • Set the number of tuners to 2 * 3 for the moment. If I get the duff tuners working, this could rise to 2 * 4, so maybe allow that to be a parameter to the program (partially done).
  • Have a hard-coded structure for the transponders (call them 1-6) and which channel numbers are on which transponder (4 per transponder) (done).
  • Hard-code in the 26 sports strings (Olympic <sport name>) and assign a rating of 1-10 to each sport too. The higher the rating, the more you want to double up the sport (i.e. record on two PCs rather than one) (done).
  • Allow for any pre-padding (2 mins) and post-padding (5 mins) in tvheadend, but add 5 seconds before and after in case two slots cushion up to each other after the padding (done).
  • Given all of the above, display a list of sports strings that need to enabled for the day on pc1 and pc2 (ones not listed should be disabled). To calculate this will be tricky - basically, every time slot mentioned in the sports data file has to be recorded somewhere at least once and doubling up will score higher for my favoured sports (but it doesn't guarantee even sports I rated 10 will get double recordings if it's a busy day).
Oh, one tip I did learn when configuring the sat tuners in tvheadend today - you have to leave "Idle scanning" ticked on if you're using the Now and Next (EIT) info like I am. If it's ticked off, it won't get the "Next" info automatically and the idea of using the sports string to insert a scheduled recording automatically falls flat on its face.

Was it just me or were the Team GB men's football team just as rubbish as ever? Maybe it was the ludicrous two noises that accompanied every single replay of tonight's match that put them off! I'm sure I didn't hear that awful noise (especially obvious with my big sub-woofer) for the Team GB women's match yesterday. After 20 minutes and 20 replay noises, I was ready to throw a brick through my plasma! I was almost hoping for no fouls or attempts on goal just to avoid any replays. A quiet swish is all that's needed if you really must have an intro/outro sound effect for replays (do you really need any effects sound during a replay?), but not a loud harsh throbbing noise that's like having your head bashed with a dustbin lid. Get rid of it, BBC!

Day 2 recording size: 387GB (a new record) - grand total so far:  603GB

Wednesday 25 July 2012

Day 1 - Women's football provides a gentle start

I'll be posting up each day until the closing ceremony, providing you with some "entertainment" as to my no doubt several recording woes. It's sad to see that the Radio Times XML feed that I was hoping to use for the 24 HD Olympics channels still hasn't got the channels listed today. It's not an issue because I've been setting up Automatic Recording rules in tvheadend - basically any BBC programme with "Olympics" in the programme title will be recorded - and all 24 HD channels have Now and Next info on them so the recording timers will be set automatically once the EIT info appears.

Yes, I have only tuned in channels that will actually air live Olympics and have split them between HD on satellite and SD on terrestrial. It should be noted that means I'll be recording BBC One HD and BBC HD on satellite and all the Freeview BBC SD channels (yes, including - but not limited to - BBC One, BBC Two and some BBC Radio channels) on terrestrial. All SD channels will be recorded twice - once on each PC - but I can only record 12 HD channels per PC now (because one sat tuner on each PC will be doing BBC One HD and BBC HD). This means I will be recording some sports once only when more than 12 BBC Olympics HD channels need to be simultaneously recorded.

This first day is handily a very light start: a maximum of three simultaneous BBC Olympics HD channels (numbered 1-3) to be recorded on each PC, plus BBC One HD, BBC HD, BBC One, BBC Three and BBC Interactive 301 get their Olympic debuts. Needless to say, the BBC have already messed up the schedule because BBC Three doesn't go 24 hours a day until Friday 27th, so BBC Three joins the Cameroon vs. Brazil women's football match at 7pm, when the kick-off appears to be 6.45pm! To compound this, BBC HD airs programmes from 4pm today and yet also joins that match at 7.00pm.

BBC HD apparently couldn't sacrifice a repeat of "Great British Railway Journeys" with the smug Michael Portillo to show Olympics football! And don't think you can use Freeview red button channels 301 or 302 to see the start of the match either - only 301 is used today and it's showing USA vs. France at the time. Got to wonder why 302 isn't used for the Cameroon vs. Brazil game really because it is indeed used for clashes in the men's football tomorrow. So it looks like Freeview viewers are already shafted on day 1 but don't worry, it may be even worse for Freeview archery fans on Friday's day 3 - I'll cover that closer to the time.

It does seem strange that with 5 full-time Olympics Freeview channels - BBC One (SD/HD), BBC HD, BBC Three and red button channels 301 (SD and HD!) and 302 - that a very light day 1 (only 6 women's football matches and overlaps only require 3 of the 5 channels) sees Freeview viewers miss out on Japan vs. Canada and Columbia vs. North Korea (including a flag "scandal" - see later) in their entirety and also the start of Cameroon vs. Brazil. That's an airing rate of about 60% for a day that they could have easily covered 100% with channels to spare. Not a good start if you like women's football and only have Freeview and it only gets worse as the Olympics progress. In other words, Freeview is not a good platform to watch or record as much Olympics as you can, but I've always said that ever since the 24 HD channels were announced.

I will be recording a few other things during the Olympics that are on the same BBC channels I've tuned in, but aren't actually live Olympics sport. I did record the Absolutely Fabulous Olympic-themed episode on Monday this week, but it got quite bad reviews so I haven't even watched it yet. I run an unofficial lottery site so I'll be recording all the lottery shows. Yes, I even bought a rare Euro Millions ticket for Friday's huge draws that are offering a record £205m in prizes to UK players. Needless to say, the BBC air the lottery show on BBC Two right in the middle of the opening ceremony, ho hum. Oh and I'll be recording the torch relay documentary (BBC One HD/BBC One 7.30pm tonight)  and Olympics countdown shows (Thursday and Friday on BBC One HD/BBC One) that are on this week.

Evening update 1:
Predictably, I had a major crisis a few hours before recording the first event. The fourth sat tuner on my first PC was refusing to co-operate, reporting poor signal quality (eventually falling to zero) and "constant FEC" status in tvheadend. As a temporary workaround, I switched BBC One HD and BBC HD recordings to terrestrial (pc2's tuners were all fine, so there was no real chance of not recording what I wanted).

I will investigate this tonight and tomorrow morning and hopefully work out exactly what's wrong. If it's a hardware fault, I can't really be without 4 tuners for an unknown number of days if I RMA the sat card, so I may have to struggle on with "only" 7 sat tuners instead of 8 until after the Olympics. And now you know why I bought two of everything - for exactly this sort of problem.

Evening update 2:
And you thought weather would be the first thing to disrupt the Games didn't you? Nope, it's the officials at Hampden Park stadium in Glasgow, who apparently put South Korean flags on the stadium screen next to player's names as they were showing pictures of the North Korean women football players! The North Koreans duly left the pitch in protest and didn't come back again for an hour. The officials apologised and I do think it was an overly exaggerated reaction to what appears to have been an honest mistake.

I've added an extra slot to catch the second half in my recordings, though the Now and Next info still says "This Is BBC Olympics 3 HD" for the 9.55pm slot, rather than "Olympic Football". Yes, I'll probably manually stop it recording once the channel returns to their timetable video when nothing's going on. It does go to show that you need to keep an constant eye on the BBC Sport website for Olympics screw-ups like this!

Evening update 3:
It occurred to me that if I hadn't checked the BBC Web site for Olympic news tonight, I'd have missed recording almost the entire second half of the Columbia vs. North Korea match thanks to the aforementioned one hour delay. What if this happens again e.g. rain delays or whatever? How do I get a recording of an event - preferably in HD - from elsewhere? I'll just say one word and leave you to Google it: get_iplayer. I've investigated as a fallback and it does work, albeit it tends to download in real-time of course.

As a final thought for the day, when BBC Three does go 24 hours on Friday, they ironically then fail to air a single second of anything Olympics - not even a preview or a documentary or anything. So that's no archery and no opening ceremony on the freshly minted full-time "BBC Olympics flagship" channel, which really does beg the question why they didn't wait another day to go to the full 24 hour schedule...

Day 1 recording total size: 216GB
(Each day I will be listing how much the "raw" - i.e. without transcoding, edits, deletion of duplicates - recording took in total across the two PCs)

Monday 23 July 2012

2 days to go and some test channel issues

Great, we're now less than 2 days away from the first Olympics sports being broadcast and the 24 BBC Olympics HD test channels have me a bit worried:
  • All the 24 test channels seem to be identical, which appears completely bonkers to me. Yep, not even the channel name is on the screen to identify you've tuned into the right channel (luckily, EIT info can be picked up to get the programme title, which is currently "This is BBC Olympics X HD" - where X=1-24 of course).
  • There is a new shockingly huge "BBC SPORT | Olympic_Rings_Image" onscreen logo in the top right now on every one of the 24 channels, which I'm hoping is just for the test channels. If that is broadcast on all (or even some) of the 24 channels when there's live sport on, it will be a national disgrace IMHO. The logo text is neither thin nor particularly transparent, it's all in capitals and, yes, "BBC" is in reverse video to make matters worse. I'm massively cringeing looking at it right now and it will seriously devalue the Olympics coverage on the 24 channels if it is used.
  • There are no speaking heads on any of the 24 channels, so you can't tell if the audio will be out of sync - that's 50% of their usefulness gone then.
  • The rotating timetable currently shown is a right hotpotch really. It again highlights that an obvious thing they should have done is air the timetable for the next set of events for this particular channel only, which would have been a hugely useful feature. Maybe they'll implement it properly when the first events take place, but the current timetable is simply rubbish (yes, it's a test, but it should really be a good test, which this isn't).
It's nice to see that digiguide.tv (which I subscribed to recently) has got the Olympics EPG, but even their data looks a bit suspect (e.g. no actual mention of 27th July's archery session and the generic sessions they have on channels 1 and 2 don't even match the 09.00-15.00 time of when the archery takes place!).

Meanwhile, the Radio Times XML TV master feed (which I use with TVheadend to schedule recordings) is steadfastly ignoring the 24 Olympics channels, which means I may have to code something myself, but even that would probably mean producing a list of programmes to manually (sigh) set the timer recording for.

Tuesday 17 July 2012

BBC publishes Olympics SD/HD satellite tuning info

Although several places on the BBC Web site claimed that the BBC Olympics SD and HD satellite tuning info would go on the logical page you expect - namely the one that's been around for years - it turns out that they gave up on that idea completely it seems.

Instead, the critical page for people like me with media centre goodies is actually a fresh one here, predictably found via a Google search and not via any link on the BBC site. It looks like they are naming them "BBC Olympics [1-24] [SD|HD]", which is at least consistent. However, everything else is "all over the place" - Eurobird 1, Astra 2A, Astra 2B and Astra 2D are all used and the frequencies are pretty scattered.

I will probably put a "-T1" to "-T6" transponder suffix on each of the 24 HD channels because although tvheadend is clever enough to "fill up" each satellite tuner that has channels on the same transponder, it's not clever enough to actually warn you in advance that you've exceeded the number of physical tuners!

For example, in my case, it could take as little as 5 simultaneous channels that are each on separate transponders for me to actually lose part or all of one of the recordings when using a quad tuner sat card. tvheadend should flag in advance all simultaneous recordings on that sat card that need to have one or more of them deleted because not enough tuners are available, but it sadly doesn't.

Sunday 15 July 2012

Streaming XBMC to phones/tablets

One nice thing about XBMC is that streaming to a new client is as easy as putting the back-end's (tvheadend) hostname, port, username and password in the client and away you go. On Android phones and tablets, however, there wasn't any official XBMC client, so I found TVHGuide instead. It's an apk (not in Google Play) that talks to tvheadend to get the channel list and presents a basic interface that passes on any stream to a suitable video player. It works fine for SD channels, but seems to fail with any HD channels, so I stopped using it.

However, the good news is that XBMC has now been ported to Android! It's apparently the full client - including add-ons - and I hope someone remembers to build it with the PVR add-ons! Quite why PVR facilities are still considered optional to the core XBMC code is completely beyond me, especially since every other client media centre software in the world seems to have them.

Of course, we're now in a race against time for the first beta apk's to appear before the Olympics start. Let's hope that they work on my HP TouchPad running CyanogenMod 9 because it will allow me to wander around the house with a tablet showing live Olympics (on a different channel to the plasmas). In the meantime, I'm playing with the official XBMC remote control app for Android, though it's missing the streaming element sadly.

Saturday 14 July 2012

BBC Olympics app - no tablet love?

I was quite excited to see this week's release of a new BBC Olympics app on the Android platform, though I  was extremely surprised to see that it was on smartphones only. Considering the app will indeed let users stream any of the 24 channels that I'll be recording, it's nothing short of astounding that Android tablets with their much larger screens can't have the same treatment. Bang goes watching it on my HP TouchPad with CyanogenMod 9 then :-(

One thing such a tablet-based app would have been good for would have been a 24-channel live video mosaic (e.g. 6 by 4 or 8 by 3) and then you just touch the appropriate channel rectangle to switch to fullscreen for that live channel (and maybe I do the same with my plasma TV setup by switching my live feed on there to match). You can't really do that sort of mosaic idea on a small mobile screen, so for the BBC not to release a tablet version is quite disappointing. Maybe they'll do a new version just before the Olympics start that does include tablets?

BTW, I've registered for the 30-day free trial of Eurosport Player on my 50" Panasonic Viera plasma smart TV today. Annoyingly, I had to put my debit card details in, despite it being a £0.00 transaction. This is done in the hope you'll forget to cancel at the end of the free trial (13th August - day after the closing ceremony), at which point they take £4.99 a month. By then, the Olympics and Tour De France will be done, so I'll cancel.

Up to 5 channels are available to pick from, but some obvious negative points are the decidedly non-HD picture quality, ad breaks, occasional "ERROR" screens on a black background that don't recover back to the normal picture automatically and the large ever-present red and white solid Eurosport logo (except during ad breaks!). An even bigger black mark to Eurosport, though, for using Silverlight to power the desktop version of their player - why not Flash or HTML 5 which have far more penetration on desktop browsers than Silverlight? Overall, the experience is only an average one - tolerable for free, but certainly not worth paying for. Plus I hate any TV channel that can't be downloaded/recorded for later viewing at a later time, especially one that's paid and with ads!

And, finally, there's a new "Post Olympics" page available where I explain what I'll be doing with the kit once the closing ceremony finishes on 12th August.

Monday 9 July 2012

Scraping the BBC Olympics schedule

You'd have thought that a downloadable CSV of the Olympics schedule would be available somewhere, but apparently not. Luckily, the BBC site has a grid view version of the schedule that is fairly easy to scrape and parse - mind you, that link was very hard to find (it's nowhere to be seen on the BBC Olympics page - duh!).

A bit of lynx and bash scripting came up with the start and end times of each sport on each day, though remember that these are split into sub-events, which often overlap and hence will require multiple HD channels. This first attempt was solely to find out the earliest start time (08:00 - Modern Pentathlon) and end times (23:50 for Beach Volleyball and the "winner" Basketball at 00:00 midnight) that the sports are due to be broadcast, ignoring any potential overruns.

Surprisingly, the athletics don't start until at least 09:00 (twice) and the earliest sessions are often 10:00 or even later. The next enhancement of the script will be dig down and find out the start and end times of sub-events, because we want to find out how many sub-events will air simultaneously (i.e. track the maximum for each day) across all sports for, say, any 1-hour period in the day. This will give a simple set of daily figures for the maximum simultaneous recordings that will be needed. In theory, it should be 24 on at least a few days you'd expect, but I wonder if this really is the case!

What may happen is that the BBC will allocate a certain number of channels per sport (and these blocks of channels may occasionally change during the Olympics because some sports take place exclusively before or after others, so you may as well re-use some blocks in those cases). We'll know later this week once 14-day EPGs start to trickle in for the 24 HD channels. I subscribed to Digiguide.tv recently as well because it looks like they're getting the 24 HD channels in their listings.

Saturday 7 July 2012

BBC official terrestrial/satellite tuning pages

A quick wander around the BBC site has found a couple of pages relating to tuning your terrestrial and satellite channels for BBC coverage (of the Olympics in our case).

First up is a pretty dismal list of the Freeview BBC channel numbers. No mention of any technical info such as the multiplex numbering/frequencies because these all vary from region to region of course. What's even worse for Olympics viewers is that channels 302 (a temporary SD red button channel for the Olympics) and 304 (a temporary HD version of red button channel 301) aren't even listed, despite both of them having test transmissions right now.

It gets even worse with the satellite list which, whilst it does have the frequency info required for manual tuning, completely fails to list the 5 "BBC Interactive" SD channels (which have been available for years!). They should be listed in the DSAT 7 section, but it also doesn't help that the BBC have given them numerical service names 6711 to 6715 rather than proper channel names either! Also note the complete lack of the various SD and HD Olympics test channels that have sprung up in the past few weeks. Still, despite this, it may be worth revisiting this satellite list regularly to see if the BBC can be bothered to update it.

Thursday 5 July 2012

BBC HD Olympics test feeds on satellite

Following on from the SD feeds, the BBC are now testing 16 Olympics HD channels - 4 per transponder - of "test slides" (excerpts from various BBC TV shows) on various Astra satellites. I've tuned into them using the KingOfSat list - find the lines with a green HD box and use the sorted SID (in tvheadend) to allocate a channel name - and a few are giving me bad signals for some reason.
And, yes, Bjork is mixed in there again!

The 4 HD channels per transponder (meaning up to 6 transponders could be used to cover 24 HD channels) does confirm that you'd need two quad tuner cards to guarantee recording every one of the HD channels at least once. It's likely most days will have less than 24 simultaneous channels broadcasting, so it is possible to double up on some sports, though trying to plan which events will be on which transponder could prove complex!

My thoughts are that you record on 3 tuners on each card (doubling up across the "important" sports if there's any spare capacity) and then use the spare tuner on each card to record the remaining BBC HD channels (BBC One HD, BBC HD and the new HD channel that cropped up recently) twice over. The terrestrial tuners can then cover all the SD stuff (BBC One, BBC Two, BBC Three, Red Button 301, Red Button 302, BBC News - and maybe Sky News?) twice over.

Yes, the SD recording will almost certainly duplicate the HD recordings, but since it only uses 25% of the disk space of the HD channels, it's worth doing in case something goes wrong with the HD transmissions. Don't believe that could happen? BBC One HD went completely off the air for 20-30 minutes during Andy Murray's live Wimbledon quarter-final match this week (I know, because my system recorded the caption shown in the BBC article) and yet BBC One SD was fine!