Some you win, some you lose...

Peter and I arrived in yesterday morning to a message informing us that one of our partners had been sending out large quantities of spam yesterday evening. The message ended: "You put this spammer out of business today or get your entire domain blacklisted tomorrow!" This could have affected all of the partners we host email for. In order to protect the hundreds of other email users who rely on us we took the unilateral step of blocking email to/from that partner till they found and fixed the problem.

We then contacted the partner concerned and they have started investigating and found that one of their computers in their office had a number of viruses, but they actually believe that the spam came from a guest from Western Europe who used their office broadband connection to do their email yesterday evening. They are still investigating. Although it was probably not the computer they found the viruses on that sent out all the spam this shows that partners MUST be more diligent with their virus protection of their own computers.

This is exactly the type of problem we were concerned about and why we blocked .DOC files in December. This is the type of thing that can happen when a computer is 'hijacked' using the zero day exploit in Word documents. We all need to consider the implications of letting guests plug in or attach via wireless to our broadband connection. Today we discussed how all of us will almost have to get to having airport style security checks for people who wish to use our connections.

Spam is getting out of hand, so we understand why the spam blocking agency took the action they did to only give us 24 hours to solve the problem: Approx 6 months ago we had about 60,000 spam messages per month to deal with. We will be dealing with over 200,000 spam messages this month. If this rate goes on as it appears to we could be dealing with over 1 million spam emails by the summer and 5 million by this time next year, by which time the real email would constitute less than 0.01% of the email we were coping with. At some stage it could become unmanageable to try to deal with!

On a lighter note, in order to disengage my brain when struggling with these issues I play a quick game of Solitaire on my phone/PDA. Today for the first time I got 2005, the first time over the 2000 level... there is only one score higher I could have got so its the second top score.

Of course getting this is totally random and has no skill at all. There are times when I feel that the fight we have with various computer problems is totally random. Strange considering they should be logical devices. What's even stranger is that often the cards seem far from random.

TWO MONTHS!

I haven't written anything since I have returned from Singapore, and suddenly I realise two months have gone by since my last entry.

Since returning from Singpore [which was almost a whole month away] I have been involved with two major projects and various discussions with our partners: The two major projects were the finishing of an interactive training DVD. This is a project that has taken over a year to complete, primarily because the project has gone through two major re-writes and was started from scratch three times in the process and partly because we were fighting faulty equipment and buggy software.

The equipment/software problem is because we don't have the funds to have the best, we are always trying to do the impossible... make media with almost no money! However, I did finish the training DVD just before Christmas, so at least I could relax knowing it was finished.

Secondly, I was trying to complete the dub of a major drama video from Arabic to Farsi [spoken in Iran]. They had done the dub in Tehran, but very badly. So we had to record the songs for the video here [we have a Farsi singer here in our town] and I went through the film sentence by sentence, phrase by phrase with another Farsi speaker synchronizing what was recorded in Tehran with the lip movements of the actors on screen. I finished the synchronizing just before Christmas but will do the mixing of the sound and adding of Farsi titles.

Then fighting the equipment began again. Because we don't have enough hard disk space in the audio dubbing suite we had to do the dub in sections and found that there was a slip in the synchronizations for each section. Then the video editing system [Premiere running on Windows] proved its old unreliable self crashing time and time again [6 times in 2 hours] which meant rebooting the system each time so nothing was action done.

The third issue that consumed time were discussions with other agencies we work in partnership with. Almost everything we do in partnership with other agencies. It's said 'no man is an island', we have no choice, we all have to work together. But this can cause major stresses if the agencies concerned don't do things the same way... there are always minor differences, but sometimes these can escalate and take time to resolve.

When we, as a family, went on vacation to see my son in Singapore I left a task for two of our Egyptian colleagues to do. The task was to configure a database system and train a team in Egypt how to use it. The programmer who was supposed to do the configuration went AWOL [absent without leave] so to speak and didn't do it. When I came back I had a very unhappy partner agency who didn't have a working database system or training.

The situation got worse as the programmer totally disappeared and we were worried he might be ill, had an accident or something. Eventually, after trying all routes to find him I got an email saying 'sorry I didn't do it and I won't be able to'. Short and sweet. So I had to work out the system and do it myself together with the other Egyptian in an online conference. We did that all Christmas Eve, finishing at 4am on Christmas Day [25th December]. Christmas Day in the Middle East is 6th January so it meant that my Egyptian colleague could train the other Egyptians before their Christmas!

As we look back over the past year and towards the new year, here are out concerns:

• What to do about the programmer in Egypt, he has been very involved in what we do over the last year and is one of the best in the region, so replacing him would be difficult.

• We lost a valuable member of the team here in November. He was our first line technical support person -- he fielded the questions before we dealt with them determining if it was a real technical problem that we should get involved with or was a problem with [for instance] the partner's computer and not our problem at all.

• Generally more people, Peter and I are both project and R&D people, we really need a team of good support people to do some of the work we are doing. We would like to train Middle Easterners to do this, but are not sure how to fund such.

• We have requests to help with maybe 3 or 4 times as much as we can actually do. If we cut down to what we can reasonably do it would leave Peter and I doing only maintaining systems which would drive us nuts as we are not maintainer type people.

• We are spending time trying to plan the next 12, 24 and 36 months to try and get out of the sort of 'Catch 22' situation we are in.

Singapore

We arrived in Singapore today. We are here for 3 weeks to spend time with Daniel. No, not work... total vacation, so no blog entries after this one for 21 days!

SPAM :-(

"We appear to be using an unusually high bandwidth on one of our two links..." that was the greeting this morning from John who is first line tech support. Hmmm... why?

20 minutes later I want to go to the balcony and SCREAM "AHHHHHHHHHHHH". Yes, I've found the culprit. Untold quantities of email are passing through one of our gateways. Thousands, tens of thousands of messages. "OK, STOP ALL EMAIL IMMEDIATELY".

We discovered a problem - one of our partners was sending out bulk email. Theoretically this was to about 1000 people who had [I hope] signed up to receive it, so it was not [I hope] spam. There was a problem in the computer program that was sending it [not written by us, but by a programmer working for them] which meant that thousands and thousands of emails were sent. Not only did this totally clog our system reducing performance to everyone, but because of the fault some people got more than 800 copies of the email [with Word document attachment]. This action could get both the partner and us listed as a spammer which has legal and ministry implications.

In response to this we have disconnected the offending partner from email entirely while we are doing whatever we can to reduce the impact of this activity upon them, us and other partners. Instantly disconnecting while we sort the problem may be irritating to them, but is essential to maintain the service to everyone. It takes time to sort out problems like this and may be a few days before we can restore service to them.

We have to very careful about sending mass or bulk emails to people. It is essential that whenever they send a mass or bulk email that you have clear evidence that the person receiving it has specifically and deliberately asked them to send it. If they do otherwise they will be classed as a spammer and their domain will become locked to send emails. What is more serious for us is that our servers will also become classed as spam servers and will then be blocked for all users to send emails.

Failing to comply is not just immoral it is also illegal under DIRECTIVE 2002/58/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 12 July 2002 concerning the processing of personal data and the protection of privacy in the electronic communications sector (Directive on privacy and electronic communications). Member states have different punishments for violation of their respective implementation of the directive, but I am sure I don't want to find out what the inside of a jail looks like without being merely a visitor!

What does the law mean in practice: If someone writes to a website or radio or TV station, then our our partners can write back and forth as much as they like. However, unless the person has specifically signed up to receive bulk emails then they may not send them an email that is sent to many people; they may not send them extra unsolicited information about the station; they may not send them a survey; they may not send anything which is more than responding to their specific email questions or comments. The response should be personal not bulk.

It is questionable and has not yet been tested in law if they may even send them an email asking them if they would like to sign up for extra information. Current opinion [among the professional ISP community] is that they may write to users who started communicating with the station before 2002 and ask them if they would like bulk emails, but they may only do this once and a non-reply must be considered to be a 'no'. In fact unless they have a specific 'yes' then everything must be considered 'no'.

As an example of the specific sign up, if they have a web page for people to send messages etc if they want them to be able to also receive bulk emails [eg newsletters or information about the radio station or website] they must have a separate check box for them to confirm they want these bulk emails. The check box must be unchecked and the user must check it - the law requires 'opt-in' not 'opt-out'. If they announce on the TV or radio an email address they must make it clear if this is for communicating with the station or to receive bulk emails. Unless they have specifically told the listener or viewer it is for bulk emails [eg "If you would like to receive ongoing information about our stations please send an email to this address"] or unless the listener or viewer has specifically asked for bulk emails [eg "Please send me information about your station whenever it is available" or "Please keep me up to date with all the programmes on your station"] then they must consider the email not to have opted in and never send them a bulk email.

Sounds severe... and it is. Currently on our main email server we reject twice as much spam email as we accept as real email and for some people the 'real' contains a high percentage of spam. Spam is a real problem, it wiill get to the point where email is totally unusable as a communication medium unless something is done about it quickly.

So that was the early morning distraction... now down to 'real work' -- mixing the audio for a drama DVD in Farsi. "Please God, give me a smooth ride on this one". This should actually be fun. I enjoy sound mixing and doing the final mix on a dub should be enjoyable.

Problem One: The Betacam tape that I am copying from appears to have problems, or is it a headclog or... phone call to engineer... no answer... try again later.

Problem Two: More major, the so called 'M and E' tape is different from the picture tape. 'M and E' stands for Music and Effects. When you are dubbing a programme into another language you prepare first an audio tape with just the Music and sound Effects. Over this you add the new dialogue and finally mix them all together to make a combined audio track. Alongside this you take the pictures from the picture master tape. The picture master tape should be identical to the M and E tape. It wasn't. Why?

Phone call to production team in Egypt. They knew there had been problems during the editing [some years ago] but didn't realise the two tapes were not identical. Now I have to find out where they are non-identical and make artistic decisions as to how to correct it. Simple... not.

Sonic Solutions or Sonic Problems?

Sonic Solutions: The leader in Digital Media... or so their website tells us. Maybe you've guessed... I'm back to problems with the infamous DVD we are making. Today I burned what should have been the penultimate DVD having corrected all the problems. There are 100 video titles, countless menus and playlists in this DVD so checking is a task in itself.

First when I came to make the DVD my dear friendly DVDit Pro came up with an error saying 'Cannot recode MPEG audio' or something very close to that message. With 100 audio titles all having MPEG audio and nearly 50% of the menus also having MPEG audio you are left screaming a question... "Which MPEG audio?"

Of course, being a computer it just sits there looking blankly at you not answering at all. Obviously the programmers at Sonic Solutions thought a guessing game was more productive to the spirit at this stage in the proceedings than actually telling us... so guess and guess again and I found the offending files.

Hallelujah... with the offending files corrected it made a DVD image. Took it home to burn. Remember the new office burner has failed and I haven't had time to pull the system apart and take it back to the shop where they would test it for 3 days, send it back to the supplier who would test it for who knows how long and eventually pronounce they will... maybe... supply a replacement.

Yes, it burned correctly. Yes, it plays correctly. No, its not perfect. Some of the corrections I did appear not to have happened. Second award for duff computer program goes to Adobe for their flagship program Premiere for video editing which when you click 'Use audio from the left channel only' sometimes indicates its doing this, plays it back through the speakers from the left channel audio but then creates a file with the left channel audio missing!

Now just in case you are wondering, please ask any Apple MAC users you know if they have the same problems with Final Cut Pro and DVD Studio Pro. When I said to a colleague from another organization that I felt like chucking the Windows based system out of a very high window she offered to help me with it! She had changed from Windows to MAC and knows my frustration. It was why she changed. While you are looking for Apple Mac users [not difficult to find they are increasing every day] go to the Apple website and watch their extremely funny adverts contrasting Windows and Mac computers.

But is everything Apple Mac perfect? No, far from it. We have a system for allowing people to send SMS messages securely to our servers. We want to have a small neat system we can ship to remote locations to enable the SMS messages to come in from all over the place. A Mac mini should be perfect. The old G4 that was available till this summer worked great. Now they introduced Intel processors and... won't work with the SMS system. Intel processors? Aren't they the processors that Windows computers use? I'm not saying a word! Silence speaks louder than words.

Well... I have 4 more days before our vacation and two things to do:

• complete this DVD
• complete a sound mix for the audio for a film dubbed into Farsi