Friday, November 14, 2008

Time Logging & Cookie Data Encryption

One of the projects I'm working on is to log how long a visitor is on our website viewing a video and use that to enable a service. Yeah... Who would of thought?

Anyway this post is both about the time logging approach utilized as well as cookie encryption. Let me give you a little background before we dive in.

The Conditions.
The goal is a little tricky because of the set up. I'm writing it in Classic ASP, (not exactly my favorite language). Also normally one would think of storing time logging data on a SQL server, but I didn't have access to one. Another tricky part about logging the time a visitor is viewing the video is that the video being viewed can be upwards of an hour.

The Challenge.
Using session to track the time data isn't a full solution here since the session will timeout unless I increase that, (which right now isn't an option). If I am to use sessions as a way to keep track of the time spent on the site, I need to continually refresh/access the page to update time logging data as well as to keep the session alive.

The Chosen Approach.
The idea was making an AJAX call asynchronously "in the background" every 10 mins while the user is watching the video. This would keep the session alive, (which I'm finding out it does not for some reason), and would allow me to track with a 10 min accuracy if the user was on our video page or not, (hopefully watching the video). The advantage here is if that if the user moves away from the page, the updates stop effectively defining when the user began and finished viewing the video.

What Actually Happened...
As much as I hate to say it, I have to admit- this system will at least partly depend on JavaScript and I can't think of any other practical way to avoid this. I find comfort in the fact that we can thank the major adoption of .NET-driven applications for JavaScript becoming a necessity such that a developer can count on a fairly large user base supporting it.

Anyway, not having access to a database, I chose to store time logging information on a cookie. I decided I would store the session ID and the time when the request was made. Of course, this could lead to easy tampering and so I chose to encrypt it.

The encrypting algorithm I used was RC4. I found it easy to implement, thanks to After reading up on it a little I also felt it was rather secure- provided a strong key is used- especially for this purpose.

Little did I know the headaches I would run into. I would encrypt the data and store it in the cookie- but when I read it back, some of the data would be missing. It was very bizarre. I was trying all these ways to make sure I was reading/writing the cookie data correctly.

Before long I realized that it probably had to do with the way cookies are stored and just exactly what is and is not legal character-wise. I tried URL-encoding them. This bloated the data and I still had data missing.

I threw the whole URL-encoding idea out the window (for the moment) and tried filtering for illegal characters myself. I did some research, but all I could find was that <>:= were the illegal characters and nothing else. So I wrote some filters for that. Still missing data. Tried URL encoding as well as the filtering. Still missing data.

Finally I decided I would just write a function to go through each character in a string of encrypted data, and find any characters outside of a safe US-ASCII code range and escape them through a custom method.

As I was finishing up I ran into the Escape() and Unescape() methods in Classic ASP- so I gave them a shot before even trying my custom functions. They worked wonderfully. On the other hand, my idea of grouping time data by session ID and then using that to calculate time spent on each session to figure out a total of time spent on the site didn't work out so well, so I am choosing another method. I might write on that later.

Lessons Learned:
When writing binary[-like] data to cookies, stick to a known safe subset of US-ASCII characters. I still don't know exactly which character it was that caused the problem, but I'm thinking maybe some odd escape character or something along those lines.

At the time of this post I'm still in the process of writing the application, so I'm open to any ideas or suggestions on how anyone else would do it.


Jhott Engineering

Introduction to Jhott Engineering:

So I was thinking today after-yet again - finding another solution to a problem that took me a long time to solve... that I should really consider writing a technical blog.

Not that I am the all-knowing software designer- far from that. In many ways, I consider myself a new programmer. However, I find that many of the problems I run into many others do as well and find themselves stuck.

Realizing much of the time the credit for being able to solve problems goes to other blogs I've read, or just a whole lot of hunting, I decided I should give back and write as well.

Who am I?

To introduce myself, I'll start with a little history:

I was intrigued with computers every since I was a young boy. I was introduced into my "computer career" when I encountered an error on my dad's computer when trying to play a video game. I was 9 years old, and this was back in the Windows 95-98 days. I decided I would start browsing through the files to fix it, so I could continue playing my game. Needless to say, I never fixed the game, but it was the beginning of my interest.

By the time I was 11-12 years of age, I got my long awaited RIS. Robotics Invention Set. I had always loved Legos, but this Legominstorms set was amazing- I could build robots! And so.. I dived into that. The set came with a visual programming software, but one day when I got an expansion set and read up on challenges, I noticed they always had the "harder" challenges for those who had the "SDK". Of course, I didn't even know what an SDK was that the time, but I was determined to find out.
Before long I ran into NQC and started programming using this C-Style language. It was great.

So now I wanted to write games! So badly! I found "Game Maker" and built a few games. That kept me going for quite a while, but it was only 2D and I wanted to write 3D Games. I played with the demo version of 3DRAD [for those of you who know what that is], and eventually BLITZ3D. Using BLITZ3D and a VB-style language I learned more and more until I was writing networking applications as well, chats, file transfers, ect.

I went to Teen Mania's Honor Academy. I wanted to work in their IT department so bad! And I did... I started writing websites. Learned HTML ina couple days, a good amount of PHP in a week, and eventually the rest came.. SQL, ColdFusion [shudder], JavaScript, etc. I also did some BI using MS SQL Reporting Services.
During my second year I wrote quite a few apps, and I was doing contract work as well. I eventually started delving into .NET as well, quickly falling in love with C#.

I currently am working for good-size ministry- I started not too long ago. There's quite a bit of opportunities to improve the systems, so that's always exciting.
I also do contract work for a couple other parties, as well as still enjoy personal programming projects and robotics. (Trying to fix that old RIS set!)


I'd love to hear any comments for improvements, suggestions, etc- I love feedback. I'm here to learn and I've got a long ways to go. I'm still young and I've got my life ahead of me. I also cannot write anything about myself without mentioning that Jesus Christ is my Lord and Savior and has been my driving force in my life since I was 16 years old.

- Jheatt.