Site

Well, I’ve been working on AE off and on since the last entry, but haven’t gotten around to entering new info here (been busy!)

The trip was interesting. I've been thinking of all kinds of things you could use AE for. Like, I was sitting in a restaurant in the mountains and wondered what others were around - users could submit to a restaurant list, rate them, do pics of them (maybe pics of the food and menu?), and then I could have requested a list of those restaurants within 5 miles, sorted by rating. Then could make a comment and rate (and take a pic, etc etc).

Also could be used for driving around a neighborhood and find out what's for rent or sale (maybe a revenue stream there? Charge Realtor to submit their listings).

The web site is almost ready to go. The Add/Find/View functions are all there and working nicely. I spent the weekend working on making it look sharp (colors, navigation, making the grid view look good and sortable, plus minor bug fixed).

I’ve also spent a lot of time getting the web off of the recordsets. Instead of calling the location data object layer, *EVERYTHING* now calls the .NET Service. This means that the same thing that serves up data to clients is the same as what the web uses. Which is REALLY good – all that stuff is in one place, so changes and such can take place there. Plus, the web site is basically now the baseline “client”. Hehehe, anyone could write the same web site, just using that service. Kinda cool, and what I’m trying to get to – the eventual goal is to be the definer of the technology (how it works and hangs, from an enterprise level), and the owner of the data. The rest of it (web sites, clients, whatever) can be written by third parties.

I’ve also finally registered www.AnnotatedEarth.com. I’m just camping the name right now, but hope by the end of the month to upgrade the package to dev on activehost (so I’ll get full .NET + SQL Server 2000)… so it will actually be LIVE! :)

For the most part, I need to test a bit more on the web, add some more web links (right now it just goes to mapquest, would like to do a few more) and fill it out (like location of the day, dev info, user setup, forums, etc.)

After that, I need to finish some clients up. I have a working MFC client (well, not working ATM as I made some slight XML/Service changes – they may not impact it, but I’m not sure). The next will be a VB, and then probably a C# .NET app (should be fairly easy, I hope, as I can rip stuff off from the web site to do that). Then I need to get it on at least one PDA. I’d like to do the Pilot, but don’t want to take the learning curve time, so will probably go with some CE-based device/pocket PC (same thing? Not sure…). I also need to get wireless locally, to test on a laptop. Hope Columbia has some wireless leaks downtown, need to check that out (or arrange for a access point somewhere downtown).

I’d like to go live with the web site soon, just to have it up and running, and to develop against. I’d also like to get a lot more locs. There’s some well know locs (like the SC capital, etc.) that I can enter. The web site itself should be usable – not really useful for the location-aware part of things, but as a straight waypoint registry, it works really nice.

I just wish I had more time to write this stuff – work’s been VERY busy the last couple of months, so I’ve only been able to work on AE on the weekends (usually to tired on weeknights to work on it). The only reason I’m working on a Monday is ‘cause I was so in the groove last night, really nailing this stuff. The big one this weekend was getting the web on top of the service. That had been a major annoyance, and what I considered a big architectural flaw.




ToDo:
- Voting
- Add hyperlink to all locs (let user enter)
- User setup
- User registration
- New “Find Loc” (have “Find Near” and just “Find”)
- Text for web
- Move group logic out of sp to location service. Allow a list to be passed into service. Validate against allowed groups for passed user, and then pass as additional part of where clause
- Add “My locations” to grpup search

Trip

I’m planning on taking a trip home to MI this Labor Day. As I’m driving, it would be a perfect time to get some GPS coordinates. I’m not sure exactly what I can have up and running by them, but I’d like to do something. Even if it’s just a local store of data on a lap top, using a hand-held GPS

LOL

I just love being able to re-use stuff. Using the powerful WFC XML wrappers I wrote for NZ, and the RS URL wrappers, I’ve been able to call my .NET GPS Service. I’m able to request a distance around a loc, and get back a list and process it, from a little MFC application. While I was pretty sure it would work, it’s nice to validate it.

Once I’ve got the XML standards down, I can focus on writing the Windows Test App (and then the GPS interface).

App

The first service is up. A loc can be passed, along with the # of feet to calculate, and a list is returned of stuff around that loc. So, in theory, I can just expand upon that and start writing client software. The input is a URL; the output is a XML document

I’ve gotten to the point where I need to formalize all the things that can be done in AE. Right now, the user ID is passed into the stored proc, and it returns all locs in that users groups. Do I want to keep it at that level? Do I want to be able to have the user get all groups, a specific group, or several groups? I need to formalize error handling, and the overall format of the XML document returns.

I’m going to write a quick WFC application to verify that the XML feed is working.

AE

Work got busy, so I haven’t had a lot of time to work on AE. I *DID* release a piece of Shareware, LEM (Live Event Monitor, http://www.ratajik.com/lem) which is very handy – I’m using it all the time at work now 

I’ve actually been doing a fair amount of .NET stuff for work. Odd… I learned .Net for AE, which I had to stop to do for work (which is going very quickly as I kinda know .NET now), which is going to help with AE (as work has made me know it much better).

This evening I put together signup, login, and the main screen (flowed around the logo, looks pretty neat). I also Improved FindLoc, and added a View Loc screen. I’ve also moved the correct selection of locs, so it limits what users can see what locs, down into the stored procs. So, when you say “find a the locs 1000 feet from x,y) the stored proc itself will only return those locs that the user is allowed to see, by user group).

ISP

Well, my Ratajik.com ISP (CIHOST) finally pissed me off to the point where I’m not going to use them in the future (for that site, or any other). The support there sucks. After looking around a bit, I realized I could get a .NET hosting site pretty easy, and, as I need to learn that for work (and I HATE old ASP/VBScript)….

I’ve moved the entire thing to ASP.NET. Right now, doing C# for the server side. I’ve spent the last couple of days moving things (didn’t take very long). Right now, you can Find and Add a loc, which is what I had running before. The basic architecture is going to remain the same.

I’ve been thinking more about what AE will allow users to do. The original thought was to have it notify the user as they got close to a know Loc. I still plan on doing that, but I might also want to be able to do the more traditional “Show me what’s around here” thing. This will then involve directions to that point (maps, routes, et al) which I *REALLY* don’t want to get into, as that’s an entire other project. I’d like to integrate into existing map software on the various platforms, which I will need to research. If I can do that (have Yahoo’s map site pop up with the route all in place and ready to go, on the PC platform for instance), then it should be fairly viable. The nice thing about just letting them know when they where basically on top of a location was mapping wasn’t really needed (“Umm…. Two steps to your left, and look under the table” *grin*).

Anyway, the decision to use .NET is a big one; one that I hope isn’t going to blow up in my face. While a lot of it it’s like stuff I’ve used before, there’s still going to be a learning curve, which will slow things down. Like, should I just use MSXML DOM stuff to I/O XML from .ASPX files, or should I move to Web Services for that part of it (remote access from applications, e.g., non-web page access). That’s basically a Web Service, so that would make sense… but I need to figure out what type of transport layer requirements I’m going to need. Having to communicate via SOAP from the Palm might (might?) be a problem. Don’t know yet, will need to figure that kind of thing out. The nice thing about straight XML is that most platforms should support it, at least current ones. And for older ones, I may need to go through a conversion service (Client->CustomStreamConverterOnWebSite->XML->Page/WebService).

DB

One idea on that table scan – there’s a coord system that can be converted to some kind of grid system (forgot the name, saw it while reading about GPS). Basically, each loc could have a Grid ID. Then, we could do a initial select on that grid (which I think can be figured out much easier than the distance thing), THEN do the distance thing inside of that grid... something or other like that.

I’ve gotten the initial DB stored proc stuff written (which was a pain. I hate ASP/VBScript. Can’t wait to go to ASP.NET. I’ve got all that stuff in VB components, but had to convert to script as I won’t be able to put COM stuff up on the initial site).

I almost went to .NET over the weekend. It appears to be a hugely superior environment over ASP/VBScript. I’d do it… but if I do, then I’m going to limit what I can put up on my domain. I *THINK* CIHost is going to allow for .Net stuff, but they won’t allow DLL’s I’m sure. I still might do it, as the basic web mechanics seem to be a lot better… have to wait and see.

Anyway, I have an ASP page that can add a loc, and another that can find locs based on feet from a passed loc. I’ve got wrapper functions that any page can call to do this (so they’re functions that wrap around the Stored Proc Stuff). Those wrappers will contain all the stuff I consider to be middle tier. If this thing takes off, all of that will move to COM (Or some .NET thing).

4/07/02

Well, buying a house impacted the amount of time I had to work on AE the last couple of weeks 

Sat down tonight and just started working through the basic web site. Finished up the tables (and connections between them), wrote a few stored procs, added the DB of ZIP codes and Country Names, and an ASP Add Location page that actually LOOKS nice, and used the two DB’s (via stored procs).

I’ve broken the site up into three conceptual parts, Web, Middle Tier, and Stored Procs. They allow for some separation, which will let me change how things work later.

The Web part is straight ASP, and knows nothing about the DB. It makes calls to the Middle Tier right now. The calls could easily be converted to COM or whatever; right now they are just server side VBScript.

The Middle Tier is also just VBScript that wraps around the data. The actual access to the data is exclusively via stored procedures.

I’m just going to work through the basic web site, get it up and running. I’ll then work on the XML and a test MFC (.NET?) application that used the site as a remote application. Then it’s off to GPS land… :P

Start

This is a log of the AE project. I actually started thinking about, and working on this a bit, over four months ago. I just recently came off a big project at work and had some time to do some actual work on it… so, I went ahead and set the development web site up and started some practical R&D.

Over the past four months I’ve primarily been working on some vaguer R&D. Looking at GPS devices and protocols, and the various platforms they run on. Also I figured out the math behind GPS (which I just implemented in practice for the first time last night).

So far the only thing that’s been done is the basic GPS distance calculation (which still has one error), the overview document, the basic web site, and a lot of thought. :P

Using something called the Great Circle calculation, I was able create a VERY funky stored procedure that, if you pass it a x/y coordinate and a distance, in feet, will pass back all of the stored GPS locations within that distance. I’m VERY happy I finally figured it out. The formula itself isn’t that bad, but implementing it in software against a large number of locations is somewhat complex.

I’m actually not very happy with the long term raw performance, but it will do for now (sub-2 second for 50,000 records in the DB, and 11 seconds for around 400,000). The stored procedure looks like:

CREATE Procedure findLocByDistance
(
@c1a float,
@c1b float,
@feet int
)
As

select * from Locationwhere
degrees(acos(
(sin(RADIANS(@c1a)) * sin(radians(Location.Latitude))) + ( cos(radians(@c1a)) * cos(radians(Location.Latitude)) * cos(radians(@c1b -Location.Longitude)))
))
* 111
* 0.621371 * 5280 <= @feet

Yeah, I know, it LOOKS simple… but it was hellish to get it working. I first worked out how to pass in two sets of x/y coordinates, and the stored proc would pass back the distance between the two. After getting that working, it was fairly simple to convert it into a select. I must admit this is the first time I’ve done trig in a stored proc  it’s also been a LONG time since I’ve done any trig at all, so this took a while to work out.

The problem with performance is it HAS to be a table scan. As it’s performing a calculation, it has to load each and every record to do the calc on. I’ve come up with some ways to make this a bit better, and the live system may REQUIRE additional constants on it (categories, state, zip code, et al.), so this SHOULD work for now. That time was also on a small box. If this ever gets to the million-record range (which takes around 19 seconds to return on this stored proc, with 5 rows) then I’ll need to invest in some bigger iron.