Category Archives: Programming

Previous Work: CSR Interface and Dashboard for MetroLINK (2012)

Static demo (w/ dummy data)

Source code (zip)

Before I begin I want to say that the code here, along with any other code I have posted, is posted with the expressed permission of the company that originally had me write it.

This is something I’ve been anxious to write about, I’m really happy with the results and it helped out quite a bit at MetroLINK.  It’s a web dashboard to show the call data from Metro’s Customer Service Representatives (CSRs).  It pulls data from an Access database through ODBC, and uses PHP to generate the main dashboard as well as the drill-down pages.  The graphs are provided by the excellent Flot javascript library.

Last year it was decided that more information had to be gathered in regards to how many calls the CSRs were taking, and what those calls were about.  Our phone system didn’t support anything beyond basic logging, so until the system could be upgraded something needed to be put in place that would allow the CSRs to track their own calls.  I opted for Access because it was a database system others were already familiar with, and I could make an interface easily enough by using VBA and Access’ own forms.  We saw results almost immediately, and had a much better insight into what the CSRs were doing.

Just using the Access built-in reporting functionality was great, but it was missing the “live” element.  That’s when I decided to start working on this in my spare time.  I discussed what we would need on a dashboard with my co-workers, and then set out to make it happen.

I had some hesitation when I was figuring out how to get the data from the Access file to PHP.  The same file was being used by the CSRs to input this same data, so I had been worried about the file being locked.  The Access forms were already split to avoid this, but I didn’t know how a connection from PHP  would behave.  With ODBC setting up the connection to the Access file was a breeze, and I was pleased to find out it handled multiple connections without issue.  On top of that I could specify that the connection was read-only, providing some security as well.

When I was designing the dashboard I wanted it to have a similar appearance to the public-facing website gogreenmetro.com, so I borrowed the color scheme and title image.  While the data was only changing on each refresh (more on that later) I wanted the dashboard to appear to have activity in it.  To get to this goal I included several hover-over effects and made things respond in useful ways where I could.  Primarily in the graphs and tables where you can highlight parts and get specific information about a point or pie piece.  While it isn’t perfect, it gives the dashboard a little more polish and makes it feel more “alive” than the same page without those elements.

After the main dashboard was completed I started working on the drill-down pages.  They can both be accessed by clicking the numbers for total number of calls and number of unresolved on the main page.  The unresolved drill-down is just a larger version of the breakdown by CSR, which is just building a table.  But the number of calls drill-down introduced some challenges.

On the main page I used the hour function to group calls by hour, and sent that to Flot.  It was simple, and worked for the purposes of the graph.  Moving on to the more advanced graphs though, that method was no longer going to work.  I had to use Flot’s time support, which means I needed to get milliseconds from the Unix Epoch, as that’s JavaScript’s native time format.  None of this was too challenging until timezones entered the picture.  Using Datediff to give me seconds from epoch gave me a sort of “false UTC” that treats the times as if there was no time zone offset.  Since the data would always be correct in the actual database and the presentation wasn’t affected, I saw no problems with this.  It actually encourages it in the Flot API instructions.

Until I checked the tooltips.  JavaScript corrects for time zone when you start using date functions, so all my times were coming in a few hours off.  PHP provides a great way to get the local time zone offset in seconds, so I used that to correct the difference by changing it before the page was rendered.  A side effect of this is that the times change depending on where the page is viewed, so 3pm Central would be 1pm Pacific and so on.  In this context it would probably be a bug, but in other contexts it would be a feature.

In all, this project taught me a lot.  It reinforced my knowledge of things like JSON, HTML/CSS, and how to implement designs to work cross-browser.  It gave me a chance to use PHP for a project, and I learned about it in the process.  Finally, it also gave me a chance to really use Flot and jQuery.  Being able to bring all these things together in one consistent project was a great experience.

Previous Work: Halo Stats (2010-2011)

 

Download the Halo Stats for TouchPad source code

When Halo Reach was released a few years ago, I stumbled upon their statistics API and saw an opportunity for a new webOS application.  I had seen some success with my Quick Subnets app and wanted to develop more, but a creative writer’s block had set in and I couldn’t find something I wanted to build.  When I saw the API that was available, inspiration finally hit!  I created an application that allowed the user to look up any Halo Reach player and see their information.

Now, I’ll be the first person to downplay the abilities of Halo Stats.  It basically only loads player and challenge data, when other applications load all kinds of per-match statistics and information, and even display it in a better format.  But at the time, deciding to write it was a lofty goal.  I had never written an application that connects to the internet before other than some class projects in college, and I didn’t really know about AJAX and JSON beyond just the concepts of them.  Writing this was an opportunity to learn both of those concepts, and through that further expand my Javascript knowledge.

One thing I remember in particular is finding out how easy it is to access JSON compared to XML or other formats.  To do this day I opt for JSON when I can because of that.  I also remember that the frameworks used on the webOS hardware would block XMLHttpRequest calls, and wanted their abstracted versions to be used instead.  That was an adventure in troubleshooting almost worth its own post!

After I had written Halo Stats for the Palm Pre, I was actually contacted by a programmer representative at Palm who wanted to get me set up to write more apps, and even encouraged me to get a TouchPad version of the application together before it launched.  The TouchPad was using a new framework called Enyo, while the Pre had used Prototype.  So at the time I was writing code for a framework with no documentation outside the HP/Palm forums, for hardware that hadn’t been released to the public yet.  All my testing was done via web browser or emulator.  It was quite the challenge, and experience!

There are things I would definitely do different if I were to write this again though.  For me the biggest problem is in the code itself, I had some problems associating the style information to the elements that Enyo was generating; so I chose instead to set the innerHTML property of the elements to some HTML I was generating, and then I would control the styling via CSS.  This was beneficial in many ways,  I could centralize my styling, it allowed me to use techniques I was already familiar with and made the development process faster.  But it was detrimental in that I had no control of the display or positioning that was happening higher up in the software, and couldn’t predict some of the output because of that.  And the resulting code now has chunks of hard coded HTML in it, which is ultimately harder to work with in the long term.  When I made MD5 Lookup I worked around that, but I had far lower styling expectations for that program.

Staying on the styling issues – looking back I really wish I had put more time into it.  I will always claim to be a web developer before a designer, but I’m not completely blind to a bad layout.  The commendations are off-center and not vertically aligned with each other, there is a blue border around the right frame for no real reason, the challenges don’t like up with the map and other elements – and I could go on.  Ultimately the design was rushed and it makes the entire application worse.  In the future that is something I be sure to avoid, by putting in the time to properly test the styling and nitpick over the small details until it looks more refined.  Again, I’m not a designer – but that doesn’t excuse a poor layout and appearance. In retrospect I’d rather have a simple design that looks great than what happened here.

At this point the TouchPad, Pre, and webOS are outmoded, and even Bungie’s stat servers only give back historical data.  No new games are being registered on their servers.; everything has been replaced with 343’s Halo Waypoint.  But, if you have some webOS hardware or an emulator image Halo Stats is still available in the app store, and you can even look up a player’s info, as long as they played before the switch over to 343. I’ve posted the source code above as well – most of my writing is in the source folder, under SplitView.js and Splitview.css.

Thanks for reading!

Previous Work: Transfer Chart at MetroLINK (2009)

This is my first post in a series detailing some of my previous work.  It serves to remind me of how I accomplished tasks before, formats and techniques I used, and gives me a means to show my work to others if needed.

When I started at Metrolink I was tasked with finding ways to improve their Computer Aided Dispatch / Automatic Vehicle Location (CAD/AVL) system and implement the data it was using.  Part of that process was filling in for dispatchers and learning the routes and stops used by the buses.

The most difficult part of this was sending passenger requested transfers.  They had to be sent from the requesting buses to the receiving buses manually, through the dispatcher.  For a seasoned dispatcher this wasn’t a problem, but in my case I never had enough time on dispatch to really cement in my mind which buses would be at each transfer location.  Eventually I found a way to make a cheat sheet:  I would use SQL to query the scheduling database, giving me an always up-to-date schedule from the current time to about an hour after that.  I would use JSP and Spring to get that information onto a web page and formatted in a way that makes determining where the transfers are going easier.  Then I could access this sheet from any browser to figure out the transfers more quickly, and give it to new dispatches to aid them as well.

Here is what it looks like, or click here for an offline demo:Metrolink's Transfer Chart

I don’t want to write pages of material about how Metro’s route system works, but suffice it to say that the Route (the colored section) is a path the bus takes, the block number next to it designates different buses on the same route, and the location shown on the right tells you where the bus will be at the time shown on top of the box.  Up top you can filter out just the routes you want to see.

A really basic example of how this would be used is this:  Assume it’s about 7:50am, and I have the screen shown above. I receive a transfer from block number 2102 requesting the Route 30.  Looking over the 8:00am entry, I can see that the 2102 is a route 10 bus that will be at Centre Station at that time.  So now I need to look at the route 30s – there is one that will be at Black Hawk College at 8:00, and another that will be at Centre Station.  The Centre Station bus is the one I want, so I’ll send the message to the bus with the block number 2302.  The whole process takes just a few seconds, which is important because there are a large number of transfers that come in.

When I was designing this I had several objectives in mind, along with making sure the chart functioned correctly.  First I wanted to keep content and presentation as separate as possible.  It makes for cleaner code – especially since this is written in JSP for actual use – and I love the idea of swapping out the CSS and some images for a complete appearance overhaul.  The only portion of this page where I break that ideal is the table up top – the background colors are set on the page.  That said, there isn’t much of a reason to change that table, and doing this via CSS is easy enough by setting an ID for each cell.

I wanted the design to be consistent cross-browser as well.  Unfortunately when dealing with IE6 there is only so much one can hope for, but generally speaking this looks the same no matter how you load it.  And it doesn’t lose functionality in any browser.  That said, since I wrote the code some display inconsistencies have popped up in the newest version of Firefox.  Specifically in how the table is handled in the top menu bar.

This project had its share of problems too, I hadn’t worked with visibility and display CSS settings before, so learning how they worked took some time and made for some unusual results.  This was exacerbated a bit when I added the ability to jump between “time points” where the buses are at one of the transfer locations.  I had to put an anchor link in an invisible div that remained active in the DOM, while not disturbing the rest of the layout, and also jumped to exactly the right position when click.  It took some time tweaking to get all of that working, but I love the results.  When you jump to an item it lines up with the top nearly perfectly.

Also, if I had to do it over I wouldn’t use Spring for this chart.  I had done some internship work at Pearson where I used Spring and JSP to display database information, and having heard about how much easier Spring makes database access I figured it would be foolish not to include it.  But this project only needed one query to be sent on load time, and all the excess that Spring brought to the table wasn’t worth it.  If I had more queries to run it would be a whole different story, but for something this simple I think Spring was excessive.

Overall, I’ve been very happy with how this project turned out and how useful it’s been.  The first day I used it I was nearly able to keep pace with the other dispatchers, and the drivers noticed that I wasn’t taking as long to get them their information.  It was even used for dispatchers in training up until about a month or two ago, when the CAD/AVL system automated the sending of transfers.  Not bad eh?

Working with webOS Enyo Web services in Google Chrome

The options described below are very useful for development, but equally dangerous. Enabling these options is a huge security risk and should not be used for normal browsing.

If you need these options enabled but still want to browse, please consider using Chromium as a development browser with these options.

I’ve been writing code for in HP’s new Enyo framework for webOS, and a constant issue has been that WebServices can only be run in the emulator.  Webservices are Enyo’s abstraction of XMLHttpRequest, basically.  One of Enyo’s biggest strengths is that it can be tested in any Webkit based browser, but when you get to the internet access portion of testing (usually the biggest part!) you have to move to the emulator.

Fortunately, there is a fix!  As noted above, do not leave this on by default as it is a huge security risk.  This disables the file access and same origin protections, allowing you to perform webservices calls in Chrome.

To do this, run Chrome with the following command line options:

--allow-file-access-from-files --disable-web-security

In windows, you can add these to a shortcut (after the quotes, if present) just be sure to make the double hyphens a single hyphen.  On OSX you can copy and paste the command line options above.  For OSX you’d want to use a script unless you want to run it from the command line each time.  If you’ll be spending a lot of time within Enyo, you may want to download Chromium and use that for development work.  Again, browsing with these options enabled is a significant security risk.

Thanks for reading!

New version of Halo Stats submitted to Palm!

A new version of Halo Stats has been submitted to Palm.  I can’t post further details at this time.

In other news, I have a project I finished last month that will be posted soon.  It’s pretty far away from code and computers, so it should make for an interesting post.  And I’m still looking for application ideas.  I really wanted to make a Yelp application for the Pre that supports checking in, but their API does not allow it at this time.

The ability to create, but no ideas on what to create. Who would’ve thought coders can get writer’s block?

Quick Subnets 1.0.5 Submitted to Palm

Quick Subnets has been updated to version 1.0.5 and has been submitted to Palm. Here is the update info:

Update in 1.0.5

– Fixed labeling for Network and Broadcast addresses. No longer labeled “Min IP” and “Max IP”

– Changed help screen to display properly on the Palm Pixi.

– Corrected error where the Network and Broadcast addresses would generate impossible IPs if a number larger than 254 was entered.

Also, Quick Subnets has had over 1500 downloads! Thanks for downloading everyone!

Quick Subnets now in Palm’s App Catalog!

Quick Subnets has been accepted by Palm!  My page for it is available in the link or in the above menu.  Otherwise you can view Palm’s page for it, where it gives you options to send it to your phone or post it to Twitter / Facebook.

Also, there have been over 700 downloads since it was accepted on 7/28/10.  I hope everyone is finding it useful!  Please send any comments / issues to QNS@john-am.com.

Making Circles in Google Charts – Undocumented Chart Types for Google Charts

So it looks like the Google Charts API has a couple of small undocumented charts in it!

There isn’t anything I can find about this written elsewhere, so here is my personal documentation for it.  The information here has been distilled from the MarkerIconCreator webpage. Where you can play with these chart types to have the URLs created for you.  I had to look all of this up for the KML generator I mentioned a few posts ago.

Circle (chart type it): Circle Example - With "X" in the middle.

Parameters: cht=it&chs=32×32&chco=FF0000,000000ff,ffffff01&chl=X&chx=FFFFFF,0&chf=bg,s,00000000

The important part to note is cht=it.  That chart type is not documented in the Google Charts API. 

Rounded Rectangle (chart type itr): Rounded Rectangle

Parameters: cht=itr&chs=32×32&chco=FF0000,000000ff,ffffff01&chl=X&chx=FFFFFF,0&chf=bg,s,00000000

Again, note cht=itr.

There are some other parts in both URLs  which are different from the documentation in how they’re used.  It’s not too hard to figure out via trial and error; but here are some quick explanations:

Color: chco=FF0000,000000ff,ffffff00

The API shows chco as series color, but it’s used a little bit differently for this chart type.  The first color is your shape color.  The second is your first shadow color, and the third is your background-shadow color.  These are regular hexidecimal colors, but with an added byte of values on the end to represent transparency.  If you are using this in a KML file, I recommend setting both shadow colors to 00.

Here are some examples with chco=00FF00,FF0000ff,0000ffAA: chco=00FF00,FF0000ff,0000ffAA chco=00FF00,FF0000ff,0000ffAA

Label Color: chx=FFFFFF

Chx doesn’t exsit on its own in the API. Other forms do, like chxt, chxr, etc.  It is the color for the label text.  It also supports adding an extra byte at the end for transparency.

Here are some examples with chx=00FFFF: Label Color Examplechx=00FFFF

Background color (gradient?): chf=bg,s,00000000

Chf is listed in the official documentation as a fill option.  Only the solid variety works here.  Then the coloring is the same Hex + transparency we’ve seen above. 

Here are the examples, chf=bg,s,00AA00FF: chf=bg,s,00AA00FF chf=bg,s,00AA00FF

Building a self-contained HTA with embedded Images and Icons.

I’m working on an HTA to generate KML files from a given CSV.  I’ve set a goal for myself to keep everything contained in the one HTA file.  Being HTML / CSS and VBScript, this is easy for the most part.  Images and icons are a little harder though. But not impossible!

For images, look at Data URI schemes.  I’ll borrow Wikipedia’s example here:

<img src="data:image/png;base64,
iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAABGdBTUEAALGP
C/xhBQAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9YGARc5KB0XV+IA
AAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAF1J
REFUGNO9zL0NglAAxPEfdLTs4BZM4DIO4C7OwQg2JoQ9LE1exdlYvBBeZ7jq
ch9//q1uH4TLzw4d6+ErXMMcXuHWxId3KOETnnXXV6MJpcq2MLaI97CER3N0
vr4MkhoXe0rZigAAAABJRU5ErkJggg==" alt="Red dot" />

It’s a bit longer than the usual <img src=”image.png” alt=”Image”> tag. But it allows for images in an HTA without additional files.  The way it works is by taking the file you want to embed, and converting it to Base64.  There are several online tools to do this, but I’ve been using this command line utility for the sake of bandwidth.

Once you have the blob of Base64 data, start an image tag with the src attribute set to “data:image/png;base64,” and append the data afterward.  The first part is needed to tell the browser what the image type is, and the encoding.  The image type doesn’t need to be PNG, any type can be used as long as the MIME type is changed accordingly.  Data URI’s work for most media types, so other things besides images can be embedded this way.

Note that because HTAs are rendered in IE, you’ll need to be aware of your target audience for this to work.  IE8 only supports data URIs up to 32KB.  IE7 and IE6 compatibility is not there, although it may be possible through this MHTML /CSS hack.  I’ll have to test it once my HTA is ready to go.

Now for Icons.  I found this piece of information on the MSDN page for the HTA Icon property.  The solution is to use the copy command, which will append the icon data onto the front of the HTA.  Before the HTML starts.

First, add this line to your HTA to specify the icon. If you have an hta: application tag already, you can add the “icon=”#”” line to it:

<hta:application icon="#">

Then, open a command prompt, and use this command, substituting the appropriate names:

copy icon.ico /b /y +original.hta originalWithIcon.hta

The + before original.hta tells the copy command to append that file onto the file being copied. With originalWithIcon.hta being the output file. /b tells copy that icon.ico is a binary file. And /y just suppresses the overwrite messages.

At this point, if you open the new HTA in a text editor you’ll see some unreadable characters before the HTML starts. That is the icon data.  With that embedded, the icon will now appear in your system tray when you run the file.  Unfortunately (at least on Windows XP) it doesn’t show up in Explorer.

Now you have an HTA with embedded images and Icons!