Homespacer>Support>Static website generation case study

spacer

September, 2012

Mergemill Pro is a versatile tool that can tackle a wide variety of data processing tasks. We recently used it for a project to explore business opportunities for developers while bringing the best benefits to clients.

Our client publishes a Christian news weekly that has an online version. For years their site was driven by a database, because the number of articles was too many to be published manually and they wanted their articles to be searchable. We created the dynamic site for them. It went online in year 2000 and worked till we were asked to revamp it a few months ago. We proposed to turn it into a static website, and suggested using Google to provide the article search facility. We were expected to automate the publishing process as much as possible.


Project Goals

When we started the project, the Alexa ranking of the site hovered around a million and there was very little search engine traffic. In front of us was an old database of about 25,000 Chinese articles in the outdated Big5 text encoding. This encoding has incomplete mapping for the common textual characters used in the publication, and therefore leads to unrecognizable characters in the online articles. Our important first task was to migrate the data to a new DBMS that supports Unicode, as well as to clean up the data and convert the encoding in the process. Mergemill Pro was our life-saver in this part of the project.

The second goal was to build traffic and grow the readership. Below was our to-do list:

Our third goal was to automate the entire Chinese web publishing process so that the client is involved as little as possible in creating the web pages. The scheduled autorun feature of Mergemill Pro was designed to meet such needs.


The Result

We set up a simple FileMaker Pro database to manage the articles and associated data, with ten scripts that automatically search for appropriate articles, sort them, and export them to specific locations. When the articles for an issue is ready to be published, all that the user needs to do is to select the ten scripts one by one on a menu. We also set up a total of 36 jobs under 4 projects in Mergemill Pro to pick up the data exports at midnight each Sunday, and to automatically do the following:

Once the jobs complete their runs, the new issue is online before Sunday 0:02 am. Regenerating the entire site of 50,000 web pages takes about 30 minutes using the current MacBook Pro, and needs to be done only once a year. When publishing the first issue of each year, Mergemill Pro also updates the Javascript that dynamically populates the year pop-up menu on all index web pages.


Regex tester for mastering regular expressions

The new site was online at the end of March, 2012. Search engine traffic began to build up daily. Five months later the average daily visits has increased over 500%, with over 60% from search engines. We asked for a brief feedback from our client, and below is what we got––translated from Chinese––from Kit Lai, their editor:

  1. We wanted the new website to be friendly to various access devices, from computers to smart phones. We also wanted the content to be in both Traditional and Simplified Chinese. These are all done.
  2. We expected the website to be easily searchable on the Web, and we find that our articles do appear on Google search results.
  3. The new site-search for articles on our site works much better than expected.
  4. The new publishing system works with more Chinese characters, especially those used solely in Hong Kong, and the new system is more convenient in creating the web pages.
  5. Mergemill Pro's greatest strengths in the new system are convenience, speed, and scheduled processing.


Solutions

:: ODBC

Our original idea was to connect to the client's FileMaker Pro content management database via ODBC, and automatically pull out the needed data at scheduled time to update the website. This proved to be not an option. We need to use Chinese in the entire process, including field and table names in the database. ODBC and SQL Select statements don't work in this case. Providing scripts in FileMaker Pro easily avoids these problems, and gives our client the control on when the new issue is ready to be published. The extra step of simply selecting the scripts on a menu is not considered a negative by our client.

If our client misses the scheduled autorun, Mergemill Pro simply regenerates the previous issue using the old exported data that are still in designated locations. When the articles are ready and exported later, the client only needs to select and run the projects manually to publish the new issue.

:: Nested Loops

Nested loops are not supported in Mergemill Pro, but it is easy to create such effect with a multi-tier setup. We did it in this project. For each issue, there is an index page with a link to each of the articles. On each article page, there is a side bar on the right containing the same index in a smaller font size.

:: Javascript Generation

The side bar on every index page provides a pop-up menu to allow the visitor to select a year number, and the links to all issues published for the chosen year will be dynamically displayed below the menu. The Javascript that does this is external, so that we only need to update it with each new issue, instead of regenerating all the web pages.

Mergemill Pro works with anything that is text. So it can process textual contents in databases and text files, such as TXT, CSV, XML, CSS, RSS, HTML, and Javascripts etc. Below is the top and bottom parts of the template for updating the Javascript mentioned above:

window.onload = writeLinkList;


function writeLinkList() {
var iOption = document.getElementById("selectYear");
var iYear = iOption.options[iOption.selectedIndex].value;
var iListLeft = "";
var iListRight = "";


switch(iYear) {
case "1999":
iListLeft = "<li><a href='1999/ti19990502.htm'>1999-05-02</a></li><li><a href='1999/ti19990509.htm'>1999-05-09</a></li><li><a href='1999/ti19990516.htm'>1999-05-16</a></li><li><a href='1999/ti19990523.htm'>1999-05-23</a></li><li><a href='1999/ti19990530.htm'>1999-05-30</a></li><li><a href='1999/ti19990606.htm'>1999-06-06</a></li><li><a href='1999/ti19990613.htm'>1999-06-13</a></li><li><a href='1999/ti19990620.htm'>1999-06-20</a></li><li><a href='1999/ti19990627.htm'>1999-06-27</a></li><li><a href='1999/ti19990704.htm'>1999-07-04</a></li><li><a href='1999/ti19990711.htm'>1999-07-11</a></li><li><a href='1999/ti19990718.htm'>1999-07-18</a></li><li><a href='1999/ti19990725.htm'>1999-07-25</a></li><li><a href='1999/ti19990801.htm'>1999-08-01</a></li><li><a href='1999/ti19990808.htm'>1999-08-08</a></li><li><a href='1999/ti19990815.htm'>1999-08-15</a></li><li><a href='1999/ti19990822.htm'>1999-08-22</a></li><li><a href='1999/ti19990829.htm'>1999-08-29</a></li>";
iListRight = "<li><a href='1999/ti19990905.htm'>1999-09-05</a></li><li><a href='1999/ti19990912.htm'>1999-09-12</a></li><li><a href='1999/ti19990919.htm'>1999-09-19</a></li><li><a href='1999/ti19990926.htm'>1999-09-26</a></li><li><a href='1999/ti19991003.htm'>1999-10-03</a></li><li><a href='1999/ti19991010.htm'>1999-10-10</a></li><li><a href='1999/ti19991017.htm'>1999-10-17</a></li><li><a href='1999/ti19991024.htm'>1999-10-24</a></li><li><a href='1999/ti19991031.htm'>1999-10-31</a></li><li><a href='1999/ti19991107.htm'>1999-11-07</a></li><li><a href='1999/ti19991114.htm'>1999-11-14</a></li><li><a href='1999/ti19991121.htm'>1999-11-21</a></li><li><a href='1999/ti19991128.htm'>1999-11-28</a></li><li><a href='1999/ti19991205.htm'>1999-12-05</a></li><li><a href='1999/ti19991212.htm'>1999-12-12</a></li><li><a href='1999/ti19991219.htm'>1999-12-19</a></li><li><a href='1999/ti19991226.htm'>1999-12-26</a></li>";
break;
case "2000":
iListLeft = ...
...
break;
case "2011":
iListLeft = ...
iListRight = ...
break;
<?Loop?><?If(Same[PubYear])?><?Else?>case "<?[PubYear]?>":
iListLeft = "<?EndIf?><li><a href='<?[PubYear]?>/ti<?[PubYear]?><?[PubMonth]?><?[PubDay]?>.htm'><?[PubYear]?>-<?[PubMonth]?>-<?[PubDay]?></a></li><?If([PubMonth]=06)?><?If([PubMonth]{2}=07)?>";
iListRight = "<?EndIf?><?EndIf?><?If(Same[PubYear]{2})?><?Else?>";
break;
<?EndIf?><?EndLoop?>

}
 
document.getElementById("linkpagesleft").innerHTML = iListLeft;
document.getElementById("linkpagesright").innerHTML = iListRight;
}

For the Javascipt update, the system only needs to export the data after 2011, and the Mergemill script in the template generates the links beginning at 2012.

spacer

Top of Page

Featuresspacer::spacerDownloadsspacer::spacerBuy Nowspacer::spacerSupportspacer::spacerTutorialsspacer::spacerTags Guidespacer::spacerSite Map


Copyright © 2001-2017 Cross Culture Ltd. All Rights Reserved.