cftopper.com

Supporting Multiple Languages

The Translation engine at work
Google API Translation at work
Over the weekend, I wrote a resuable script that converts snippets of English text to 20 other languages using the new Google translate Ajax API for our Teamwork Project Manager software.

The script was hacked together quickly and isn't the best most structred code in the world but it works well - it cycles through any strings in the languages table that haven't been converted yet and one by one, converts them to a foreign language. Every time, the script has 20 results, it saves the changes to the database.

Watching it at work is pretty cool with strings coming back in Arabic, Chinese, Russian etc.

The strings are stored in the following table MySQL table:

CREATE TABLE `languagestrings` (
  `languageStringId` int(10) unsigned NOT NULL auto_increment,
  `languageStringRef` varchar(255) collate utf8_bin NOT NULL,
  `languageCode` char(2) character set utf8 NOT NULL,
  `languageString` text character set utf8 NOT NULL,
  `languageStringUsageCount` int(10) NOT NULL default '0',
  PRIMARY KEY  (`languageStringId`),
  UNIQUE KEY `langStringRef` (`languageStringRef`,`languageCode`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

And translating the strings is done with the function
translate( ref, english_default, bUpdateNow, language ) which works like so:

#APPLICATION.teamworkpm.translate( "Your Projects", "Your Projects", false, "ES" )# gives "Su Proyectos".

Here is the translate function code:

<cffunction name="translate" returnType="string" output="No">
<cfargument name="ref" type="string" required="yes">
<cfargument name="englishDefault" type="string" required="Yes">
<cfargument name="updateNow" type="boolean" default="no">
<cfargument name="lang" type="string" default="#SESSION.lang#">
<cfset var getLangString = "">
<cfset var insLangString = "">
 
<!--- Ensure there is a structure for the currency language code --->
<cfif NOT StructKeyExists( APPLICATION.langStrings, ARGUMENTS.lang )>
	<cfset APPLICATION.langStrings[ ARGUMENTS.lang ] = StructNew()>
</cfif>
 
<cfif NOT StructKeyExists( APPLICATION.langStrings[ ARGUMENTS.lang ], ref )>
	<cfset ARGUMENTS.updateNow = true>
</cfif>
 
<cfif ARGUMENTS.updateNow>
 
	<!--- Check to see if the value is in the database --->
	<cfquery name="getLangString" datasource="teamworkpm_master">
	SELECT	languageString
	FROM	languagestrings
	WHERE	languageStringRef = <cfqueryparam value="#ref#" cfsqltype="CF_SQL_VARCHAR">
			AND languageCode = <cfqueryparam value="#ARGUMENTS.lang#" cfsqltype="CF_SQL_VARCHAR">
	</cfquery>
	
	<cfif getLangString.recordCount>
 
		<cfset APPLICATION.langStrings[ ARGUMENTS.lang ][ ref ] = getLangString.languageString>
		
		<cfquery name="updateLangStringCount" datasource="teamworkpm_master">
		UPDATE	languagestrings
		SET		languageStringUsageCount = languageStringUsageCount + 1
		WHERE	languageStringRef = <cfqueryparam value="#Left(ref,80)#" cfsqltype="CF_SQL_VARCHAR">
				AND languageCode = <cfqueryparam value="#ARGUMENTS.lang#" cfsqltype="CF_SQL_VARCHAR">
		</cfquery>
		
	<cfelse>
		
		<!--- Store the value in the field --->
		<cfset APPLICATION.langStrings[ ARGUMENTS.lang ][ ref ] = ARGUMENTS.englishDefault>
 
		<!--- Insert the default english value into the database --->
		<cfquery name="insLangString" datasource="teamworkpm_master">
		INSERT INTO languagestrings( languageStringRef, languageCode,
		languageString, languageStringUsageCount )
		VALUES( <cfqueryparam value="#ARGUMENTS.ref#" cfsqltype="CF_SQL_VARCHAR">,
		<cfqueryparam value="#ARGUMENTS.lang#" cfsqltype="CF_SQL_VARCHAR">,
		<cfqueryparam value="#ARGUMENTS.englishDefault#" cfsqltype="CF_SQL_VARCHAR">, 1 )
		ON DUPLICATE KEY UPDATE languageString =
		<cfqueryparam value="#ARGUMENTS.englishDefault#" cfsqltype="CF_SQL_VARCHAR">
		</cfquery>
 
	</cfif>
 
</cfif>
 
<cfreturn TRIM(APPLICATION.langStrings[ ARGUMENTS.lang ][ ref ])>
 
</cffunction>


You can see that if the string has already been looked up, the result is returned immediately so that this function is nice and fast.

If a string doesn't exist for a given language, the default english version is inserted for that language, so that it can be translated later.

The languageStringUsageCount is updated whenever a string has to be looked up, usually after a server reboot. This will be useful when we want to strip the database of strings that are no longer used in a few months.

The "update now" argument allows programmers to quickly change the value of english strings directly from the code base during development.

Hopefully this will help you get your own multi-language systems up and running in ColdFusion.
Tags: ColdFusion | Tips | WebDev

Backup Software Recommendations

I'm going to tell you a secret, but don't tell anyone. Shhhhsss.
About 3 weeks ago on a quiet Tuesday evening I installed the latest version of subversion client TortoiseSVN on one of our website hosting servers in Dublin - I had to reboot the server after the install. This is when it all went wrong...

I usually wait maybe 2 minutes and the server is backup serving a plethora of client websites and a few small web apps. I waited and waited, repeatedly pinging the server, searching for the first sign of life.

The server was apparently lifeless. Nada. Nothing. Zip. "No reply from server"...

We called our hosting company. No I lied ... we didn't. We actually had to use their brutal support chat program and bludgeon English into a pigeon-English speaking Polish guy to request that the server be manually restarted.

I guessed that I had hit the "shutdown" option instead of the "restart" option and that the technicians would reboot the server in a jiffy and that all would be well.. so I tottered off home.

Some time later I got a frantic phone call from Dan informing me that the technicians said the server's hard drive was fried. I turned visibly white and raced back to the office for damage assessment.

Hours later, the technicians in Dublin had managed to salvage some data by plugging the hard drive into another PC and running a Linux program to do a sector by sector copy where possible. We recovered a lot of data but we had still lost a fair amount. Luckily we had most, but not all, of the website databases backed up to a PC in my office thanks to Navicat and scheduled tasks.

The technicians reinstalled a blank Windows 2003 server installation along with the data they managed to recover and Dan and I spent the entire night working feverishly to get all the websites and applications working before our clients got to work and started screaming.

In the end, the only thing we lost was a very small website. We recovered the page content from Google page cache (isn't Google great) and then gave the client a new, much better website for free to make up for the down time which was 1 day while Dan knocked the new site out.

And here's my point: disasters will happen, so plan for them.

Our New Backup Plan


We have since installed Carbonite Backup on all our servers. Everywhere. It's an absolutely bargain for only $30 a year per server for unlimited data. I thought there must be a catch at first, but no, it's a steal, easy-to-setup and works great.

That's the files sorted. But what about the databases. Well, we use MySQL on all our servers (if you use anything else you are a stupid Microsoft whore or Oracle slut, sorry) and it's difficult to restore MySQL from just the raw data files. I wanted something that could backup an entire server of MySQL databases automatically including new databases without any additional configuration - the reason we lost that one website was because each website had to be manually configured with our old database backup system and we had missed a few.

Eventually, after a serous amount of poking about ye olde internet, I found exactly what I was looking for in Auto Backup for MySQL from Swordsky SoftWare. This is the only program I could find that would run as a service and backup multiple entire servers reliably without needing additional configuration as each new website is added.

It also has a thoughtful array of options such as automatic removal of old backups after so-many days - something that I used to have to do manually in the past. The SwordSky team, whoever they are, have done a fantastic job here and I encourage you to give it a go. It's a bargain at only $100 or so too.

So there you have it, two cheap, easy-to-setup products that may one day save your ass.
Tags: Tips | Tools | WebDev

About Topper on ColdFusion

Peter Coppinger aka Topper is a neurotic web monster who spends most of his chaotic life developing ColdFusion web applications when not drinking himself into a stupor and scheming his plans for world dominance.

Peter founded Digital Crew way back in 1999. Digital Crew run CFTagStore.com and have also produced lots of powerful ColdFusion tools like ProFlashUpload and CFMyAdmin.

I made this site to share my thoughts, tips and tools with fellow ColdFusion developers.

If your a ColdFusion developer, go ahead and subscribe to this site and in exchange i'll try to provide quality content to make it worth your while.
RSS Feed for Topper on ColdFusion

    I'm speaking at CF-United Europe!

    CFDevCon I'm going to be speaking at CFDevCon08! It's my second time speaking in front of more than 10 people so please lend your support.

    The topic is:
    Introducting TeamworkCMS and Site Engine - Building better websites in half the time or something like that..

    Digging

    My Work - Just Finished

    • modules.cit.ie
      Web-=based modules/programmes designer tool and database system for Cork institute of technology.
    • Teamwork Project Manager
      The top secret project is finally released. The project management app will rock your world - give it a go.
    • PMG
      New website for Project Management Group website.
    • Digital Warehouse Wholesale
      Added wholesale products to existing client website.
    • New Digital Crew documentation website
      New version of documentation.digital-crew.com using new InfinityCMS site engine. It's done now. Just add content.
    • PFH Company Webite
      New website/CMS/Newsletter System for prestigious Irish IT company.
    • Module Manager for CIT
      CIT is switching to module based courses. We are making an application for managing/submitting these modules. Gettig there.
    • Bons Secours Cork Hospital Intranet
      New Intranet for Bons Secours hospital in Cork. Considering turning this Intranet system into stand-alone product.
    • Revamping InfinityCMS
      I'm making major improvements to our content management solution, InfinityCMS. Making it faster, more powerful and easier to check into/out-of source control. Done but it's always going to be evolving.
    • BPC Update
      Minor functionality update for internal Pfizer Best Process Chemistry project.