When I started at the BBC all the news was shot on film. The first sequences I directed were on film. Film is a nice medium - when you hold it up to the light you can see little pictures and with a magnifying glass you can see exactly what you have. Between each picture there is a nice little sprocket hole for the projector [or telecine machine in television] to align the each picture.
The ongoing saga of making DVDs makes me long for the 'good old days' of film. With all this 'top field' 'bottom field' stuff it is complicated and invisible. However, today I have cracked it [I hope]. Although DV tapes are recorded with bottom field first, somewhere in the import process this gets converted to top field first... but... Premiere seems to think it hasn't so says 'export this bottom field first'. I don't know how many places you can change the field order... too many for my liking, but at last I have found the right combination and I am busy making a final trial DVD.
Wednesday, August 02, 2006
Switch over
So why did clustering not work? We [two of us] tried everything we could for a week to get clustering to work. Sadly we couldn't make it work with the resources we had. Reading what other people were doing online with clustering I found that most people had 30 Gig RAM or more in their servers [we have 1 Gig in ours and many home machines have half that].
To get that amount of RAM you need four 8 Gig RAM cards. Each 8 Gig RAM card costs approx 11,000 US dollars... yes, you do did read that right, so to have 32 Gig in two servers would cost just under 90,000 US dollars, that's just under 50,000 pounds sterling! So, assuming the price would drop within the lifetime of the servers we investigated... only to find our servers would only take 4 Gig RAM and to get more powerful motherboards would cost another silly amount of money... so forget that.
So we then spent two more days rebuilding the system to be a 'replication pair' of servers. This is not as powerful as a cluster but works well with the resources we have available. And we have now switched over to the new system.
Alongside this, the video editing project needed a new DVD authoring program - that's the software that enables menus and other clever things to be added to DVDs. The new software refused to work properly, producing very jittery images. I tried all things to make it right and nothing seemed to work. So... reluctantly I phoned technical support in the USA. This is a charge service so I was trying to do every option before incurring extra costs.
Tech support was not very helpful. Having asked all the options about what I had tried they then suggested that the display adaptor software on the computer was out of date [we had the latest software] or maybe the DVD burner needed new firmware [we had the latest firmware] or maybe the DVDs themselves were bad [we have used them successfully for months] - everything but a problem with their software of course. Not helpful.
This morning I woke up thinking maybe there was a problem with the GOP structure of the files. Oh, yuk this is really technical! GOP stands for the highly technical phrase 'Group of Pictures' and what happens on a DVD is that every so often they have a complete picture but within the 'group of pictures' they only have the differences between the pictures. This saves a huge amount of disk space. Hmmm... but... the GOP structures were correct.
I then read somewhere online that somehad had a problem with the 'field dominance order' of some video capture cards... I wonder??? 'Field Dominance Order' really is too technical to explain simply. But basically normal TV has the top field first but DV tapes have the lower field first. OK, strangely... very strangely... very strangely indeed... it appears that if I invert the field dominance order in the program making the video files and then invert it again [ie put it back where it started] in the program making the DVD it works! But... the quality is not quite what is needed as I had to convert the file to Quicktime and back again. So... more investigation.
Third breakthrough... Peter had been struggling with some computers in our 'lab' which were refusing to work. They are regular desktop computers, but he found today that if he turned off the battery saving system [normally only of use on laptops] they work correctly. As the Americans say 'Go figure'.
So we move on... this is really boring but necessary stuff.
To get that amount of RAM you need four 8 Gig RAM cards. Each 8 Gig RAM card costs approx 11,000 US dollars... yes, you do did read that right, so to have 32 Gig in two servers would cost just under 90,000 US dollars, that's just under 50,000 pounds sterling! So, assuming the price would drop within the lifetime of the servers we investigated... only to find our servers would only take 4 Gig RAM and to get more powerful motherboards would cost another silly amount of money... so forget that.
So we then spent two more days rebuilding the system to be a 'replication pair' of servers. This is not as powerful as a cluster but works well with the resources we have available. And we have now switched over to the new system.
Alongside this, the video editing project needed a new DVD authoring program - that's the software that enables menus and other clever things to be added to DVDs. The new software refused to work properly, producing very jittery images. I tried all things to make it right and nothing seemed to work. So... reluctantly I phoned technical support in the USA. This is a charge service so I was trying to do every option before incurring extra costs.
Tech support was not very helpful. Having asked all the options about what I had tried they then suggested that the display adaptor software on the computer was out of date [we had the latest software] or maybe the DVD burner needed new firmware [we had the latest firmware] or maybe the DVDs themselves were bad [we have used them successfully for months] - everything but a problem with their software of course. Not helpful.
This morning I woke up thinking maybe there was a problem with the GOP structure of the files. Oh, yuk this is really technical! GOP stands for the highly technical phrase 'Group of Pictures' and what happens on a DVD is that every so often they have a complete picture but within the 'group of pictures' they only have the differences between the pictures. This saves a huge amount of disk space. Hmmm... but... the GOP structures were correct.
I then read somewhere online that somehad had a problem with the 'field dominance order' of some video capture cards... I wonder??? 'Field Dominance Order' really is too technical to explain simply. But basically normal TV has the top field first but DV tapes have the lower field first. OK, strangely... very strangely... very strangely indeed... it appears that if I invert the field dominance order in the program making the video files and then invert it again [ie put it back where it started] in the program making the DVD it works! But... the quality is not quite what is needed as I had to convert the file to Quicktime and back again. So... more investigation.
Third breakthrough... Peter had been struggling with some computers in our 'lab' which were refusing to work. They are regular desktop computers, but he found today that if he turned off the battery saving system [normally only of use on laptops] they work correctly. As the Americans say 'Go figure'.
So we move on... this is really boring but necessary stuff.
Thursday, July 27, 2006
Up periscope...
"Up periscope... scanning horizon... clear to surface..."
This last month seems like 2-3 months in one month. Mainly we moved house, which we had intended doing earlier in the year when it was cooler, but happened in July. The house move went well and can be read in detail in Sue's blog, but it did take me away from other work for quite a while.
Then there was editing to catch up on. This training DVD just won't go away...
Then we have a major rebuild of the servers. One of my colleagues from Egypt has come over for the month so that Peter, he and I can all work together so that we are all understanding the system together.
What we wanted to do was an upgrade to one of the database systems we host to stabilize it for 3 years - obviously minor upgrades during that time, but the major one now. Within that upgrade we wanted to have automatic backup of the data and automatic changeover between the servers. This is called clustering. Another change had been in process by the programmer of the system and that was to incorporate all the images into the database.
Everything worked well until the day we were due to switch over from the old system to the new... when we tried it we found that one of the databases was very large indeed and clustering required more resources that we have available, but that the newer version of the database software would probably handle the large database better. So now we are in investigation for how to proceed as the databases will grow significantly during the 3 years we want this system to work for.
Alongside this, there is the war in Lebanon. We hadn't anticipated getting our guest apartment ready yet, but had to do so in a hurry to accomodate a couple who had evacuated Lebanon.
This last month seems like 2-3 months in one month. Mainly we moved house, which we had intended doing earlier in the year when it was cooler, but happened in July. The house move went well and can be read in detail in Sue's blog, but it did take me away from other work for quite a while.
Then there was editing to catch up on. This training DVD just won't go away...
Then we have a major rebuild of the servers. One of my colleagues from Egypt has come over for the month so that Peter, he and I can all work together so that we are all understanding the system together.
What we wanted to do was an upgrade to one of the database systems we host to stabilize it for 3 years - obviously minor upgrades during that time, but the major one now. Within that upgrade we wanted to have automatic backup of the data and automatic changeover between the servers. This is called clustering. Another change had been in process by the programmer of the system and that was to incorporate all the images into the database.
Everything worked well until the day we were due to switch over from the old system to the new... when we tried it we found that one of the databases was very large indeed and clustering required more resources that we have available, but that the newer version of the database software would probably handle the large database better. So now we are in investigation for how to proceed as the databases will grow significantly during the 3 years we want this system to work for.
Alongside this, there is the war in Lebanon. We hadn't anticipated getting our guest apartment ready yet, but had to do so in a hurry to accomodate a couple who had evacuated Lebanon.
Sunday, June 18, 2006
One of the services we provide is what is called a 'contact management system' for people who contact various radio and TV stations. 'Contact Management' is a way of ensuring that letters, emails, SMS messages and phone calls are all dealt with promptly and that the people doing this can know what contact has happened with an individual viewer or listener before. The radio and TV stations we help have tens of thousands of contacts per year, so this kind of thing is important. It is also important that the information is kept private and confidential.
When we started providing this service about just under two years ago the numbers on the system were pretty small, but have grown out of all proportion and now the two servers we had handling this system were really not coping very well. Every time one of the people doing listener support would click on a page of information it could take up to 30 seconds to display. They were getting very frustrated. As a clue the BBC home page takes about 7 seconds to display everything... this was a text only page and taking 4 times as long!
This upgrade has been long in the works, if you have followed th saga of the motherboards you will know it well. But at last this week we managed to upgrade the main server and at last all the parts have arrived to upgrade the backup server.
One of the things we also do is research new technologies to see how they can be used for communicating around the region. This week Peter showed me something that as they say 'blew my mind away'. We have been streaming audio for some time and doing trials with video streaming, including streaming video to some pretty large numbers of people from various conferences. But to my mind video on the Internet is pretty lame. If the material is not available elsewhere then I guess it's OK to watch but I'd much rather watch satellite TV.
However... Peter showed me the new Reuters website with embedded video. The quality was stunning and smooth [often Internet video is jerky] with good sound. The technology used for this is Flash video and uses Flash Media Server software.
Funny enough this has come along just at the time we need it for a project we have discussed for some time, and has grown closer to actually happening while I was in Egypt. There is a TV programme on one of the satellite TV channels about a couple of small groups of people who each meet together in their homes to worship Jesus. As you can imagine in some places in the region this is the most appropriate way for followers of Jesus to meet.
The desire is to create an Internet 'home' that people can come to who see the TV programme and who want to know more and want to link up with others who feel the same as they do. I use the term Internet 'home' because the aim is to be much more than a website, with lots of things on offer and eventually live linkups between people who cannot meet physically. We had already decided to use Flash technology, but the Media Server technology will make it much more open and friendly for those coming to the site. Of course dreaming the idea is only 1% of the work!
When we started providing this service about just under two years ago the numbers on the system were pretty small, but have grown out of all proportion and now the two servers we had handling this system were really not coping very well. Every time one of the people doing listener support would click on a page of information it could take up to 30 seconds to display. They were getting very frustrated. As a clue the BBC home page takes about 7 seconds to display everything... this was a text only page and taking 4 times as long!
This upgrade has been long in the works, if you have followed th saga of the motherboards you will know it well. But at last this week we managed to upgrade the main server and at last all the parts have arrived to upgrade the backup server.
One of the things we also do is research new technologies to see how they can be used for communicating around the region. This week Peter showed me something that as they say 'blew my mind away'. We have been streaming audio for some time and doing trials with video streaming, including streaming video to some pretty large numbers of people from various conferences. But to my mind video on the Internet is pretty lame. If the material is not available elsewhere then I guess it's OK to watch but I'd much rather watch satellite TV.
However... Peter showed me the new Reuters website with embedded video. The quality was stunning and smooth [often Internet video is jerky] with good sound. The technology used for this is Flash video and uses Flash Media Server software.
Funny enough this has come along just at the time we need it for a project we have discussed for some time, and has grown closer to actually happening while I was in Egypt. There is a TV programme on one of the satellite TV channels about a couple of small groups of people who each meet together in their homes to worship Jesus. As you can imagine in some places in the region this is the most appropriate way for followers of Jesus to meet.
The desire is to create an Internet 'home' that people can come to who see the TV programme and who want to know more and want to link up with others who feel the same as they do. I use the term Internet 'home' because the aim is to be much more than a website, with lots of things on offer and eventually live linkups between people who cannot meet physically. We had already decided to use Flash technology, but the Media Server technology will make it much more open and friendly for those coming to the site. Of course dreaming the idea is only 1% of the work!
Wednesday, June 07, 2006
Just back from Egypt where I was directing a Live Outside Broadcast... and there are a lot of pretty upset people back there in Cairo. Why? People from the event we were supposed to be broadcasting kept coming in to the video control room where we were working from and asking us to feed the programme onto the screens so everyone could see it but we could not because it wasn't there on the satellite channel it was supposed to be on... there were many phone calls asking why it could not be seen on the satellite channel... the people that sponsored the uplink kept asking why they could not see the programme they were paying for. Just as a clue, the cost to the sponsors for this one Live OB was the equivalent of half the price of buying a flat to live in. So you can understand why they were upset. Why wasn't it on the satellite channel? I have no idea. Someone somewhere decided it should go out all over sub-saharan Africa but not the Middle East.
However, for me personally, I also was upset about the communications between our OB site and the main Outside Broadcast which meant that for the second year running we had a disaster uplink. Before the event I kept repeating to the broadcast organizers that the communications needed to work well and I was concerned as soon as I heard what they were attempting to use [one ordinary phone line] - that they were totally inadequate.
In the morning of the Outside Broadcast we tested the communications, connected the phone line into our talkback and although the quality wwas poor it were adequate and I was encouraged. Maybe it would work and I would be proven wrong.
When it came to the live OB we had problem after problem connecting and when we did it appeared that the OB engineers in at the remote site had no clue how to handle the facilities. When we first connected the sound was adequate and we could hear the director, then it became distant like down a tunnel... it sounded like we were on a regular phone handset that had been placed on the desk, not connected into the talkback system, but I cannot believe anybody would even attempt a TV OB using a phone handset for communications, so I am not sure what was happening. Then after a while we would get tone dialing on the line and then the line would be hung up. We would then redial and redial and redial... for 30 minutes or longer only getting voicemail, eventually connecting only to repeat the performance. Taking that long to re-dial each time meant we missed the first two inserts we were supposed to do for the broadcast.
It is almost 30 years ago to the month when, as a very junior Outside Broadcast engineer, I was involved in sound mixing a major OB for the BBC. Over a period of 10 days we did numerous inserts into radio stations around the world including many in Africa. Some were simple but many were more complex than the OB last weekend.
I remember particularly doing a sequence with Kenneth Kaunda simultaneously into a number of radio stations including Zambia. It was notable to me because part of the interview included this African statesman sharing his faith in Jesus as being the only hope for the continent.
Why I am now upset is that as an 18 year old junior OB Engineer I set up and facilitated many links with complex communications over those 10 days, some difficult to achieve but all successful. Since then, over the years I have been involved in many Christian events, including directing significantly more complex ones than the OB last weekend. Now I weep because we had absolutely appalling communications again. And it was totally unnecessary to be that bad.
Of course I also had my own traumas in Egypt again, where they swapped the second unit director to someone who had never directed TV before and therefore got really bad pictures and blocked my requests for backup interview guests to cover if there were problems - eg when we hit a time of total silent prayer for the only time we did get on air... but that's another story...
It is the second year running there have been problems. The first year I just felt sorry for the organizers thinking it was a unique occurence. This year has been a repeat performance. So what do I do? At the moment I have said I will not spend my time on these Live OBs in the future. Should I spend more time teaching more people? Will they learn? Will I really learn to say NO!
However, for me personally, I also was upset about the communications between our OB site and the main Outside Broadcast which meant that for the second year running we had a disaster uplink. Before the event I kept repeating to the broadcast organizers that the communications needed to work well and I was concerned as soon as I heard what they were attempting to use [one ordinary phone line] - that they were totally inadequate.
In the morning of the Outside Broadcast we tested the communications, connected the phone line into our talkback and although the quality wwas poor it were adequate and I was encouraged. Maybe it would work and I would be proven wrong.
When it came to the live OB we had problem after problem connecting and when we did it appeared that the OB engineers in at the remote site had no clue how to handle the facilities. When we first connected the sound was adequate and we could hear the director, then it became distant like down a tunnel... it sounded like we were on a regular phone handset that had been placed on the desk, not connected into the talkback system, but I cannot believe anybody would even attempt a TV OB using a phone handset for communications, so I am not sure what was happening. Then after a while we would get tone dialing on the line and then the line would be hung up. We would then redial and redial and redial... for 30 minutes or longer only getting voicemail, eventually connecting only to repeat the performance. Taking that long to re-dial each time meant we missed the first two inserts we were supposed to do for the broadcast.
It is almost 30 years ago to the month when, as a very junior Outside Broadcast engineer, I was involved in sound mixing a major OB for the BBC. Over a period of 10 days we did numerous inserts into radio stations around the world including many in Africa. Some were simple but many were more complex than the OB last weekend.
I remember particularly doing a sequence with Kenneth Kaunda simultaneously into a number of radio stations including Zambia. It was notable to me because part of the interview included this African statesman sharing his faith in Jesus as being the only hope for the continent.
Why I am now upset is that as an 18 year old junior OB Engineer I set up and facilitated many links with complex communications over those 10 days, some difficult to achieve but all successful. Since then, over the years I have been involved in many Christian events, including directing significantly more complex ones than the OB last weekend. Now I weep because we had absolutely appalling communications again. And it was totally unnecessary to be that bad.
Of course I also had my own traumas in Egypt again, where they swapped the second unit director to someone who had never directed TV before and therefore got really bad pictures and blocked my requests for backup interview guests to cover if there were problems - eg when we hit a time of total silent prayer for the only time we did get on air... but that's another story...
It is the second year running there have been problems. The first year I just felt sorry for the organizers thinking it was a unique occurence. This year has been a repeat performance. So what do I do? At the moment I have said I will not spend my time on these Live OBs in the future. Should I spend more time teaching more people? Will they learn? Will I really learn to say NO!
Subscribe to:
Posts (Atom)