Working on a remote team

 Scott Hanselman gave my team a great talk today!

He's been a remote worker for a long time.  I've also been remote over the years, and have also led mixed remote/in-person teams, so am very interested in his ideas around that. I've found leading those mixed teams can be challenging. 

He blogs about this a lot - Here are some of the articles he's written that resonated with me on remote work

Tools

Looks like SQL Server 2008 has some Geospatial stuff that I could leverage to do some of this (Gawd I could have used this stuff back in 2003!). I've also been poking around at Bing and Mappoint, which have some pieces that I could use (but I don't see supporting what I need, as far as custom data goes - it's too limiting).

I'm also trying to figure out if Azure has any of this new stuff. At this point, it is my preference (and my hosting company only has SQL 2005).

Might be able to develop the rough grid stuff off of this:

• Equirectangular projection
• Universal Transverse Mercator coordinate system
• Maidenhead Locator System

The Maidenhead looks promising - smallest resolution is 3.5 x 3 miles - and the encoding (MM44NN55) would easily lend itself to very fast SQL lookups (Quick calc to convert lat/long to Maidenhead, and then could pick which pair to look at - so MM for the world-level grid, 44 for regional, NN national, and 55 for local).

And...

I could have two calcs here - if a fire is below a certain strength (90% of them will be, I think), they won't even be on a grid.

So do the calc of all fires within, say, 5 miles. Then do a calc using the combo technique below. So a majority of fires never get a grid entry, and would be calc'd using a simple Great Circle calc to figure out effect. The other ~10% would use the grid system to figure out which of them would have a effect. And... the top level wouldn't really need to be a grid here, as I'm not calcing based on distance (on the grid cells) - "Global" effecting fires could just be marked as such. Every requested temp calc would include a calc on them.

Combine?

So it occurred to me that I could combine both of these together - processing each "fire" to figure out any given spots given temp, and pre-calculating a grids ambient temperature when a fire is placed.

The problem with calculating what fires effect a specific spot is it could by ANY fire, any place on the earth, if the fire is big enough. If I have a million fires, even if only 500 of them effect a specific spot, because of the range-calc of each fire, I don't see any other way, using this technique, to figure out the temp then get get all million can calc which is having a effect.

Using the grid makes it much simpler - instead of looking at individual fires when a locations temp is calculated, when the fire is placed, all grid locations that that fire will impact have their temp recalculated. Several layers of grids work together on this and a fires effect is only calc'd on the largest possible grid.

So, combining these two - when a fire is placed, figure out the appropriate level of grid the fire will effect (Global, regional, local, small, etc) and instead of pre calc'ing the temp on that grid, in those cells, just add a indicator that the fire effects that grid (and cells). When a calc is requested, find all fires that effected the current grid(s) of the current location, and then do a calc against ALL fires that have entries in that grid.

This has two very large benifts that the first grid techinque didn't have - once the list of fires is know, a real distance calc (location of both calc location and fires location and radius of effect of fire) can be made - so this gets us back a true gradiant. It will also allow a fire to expire or reduce over time (based on the current date, on the calc modify the heat level of the fire by how old it is and how strong it intiall was).

So this Combo calc will have a lot more overhead, but by combining the two techniques, I think it's do-able - and gets what I really wanted to stat with in the first place.

More thoughts

So here's a visual of the optimum way of doing this, simplified (the values radiating from the center of each fire would really be something like 15, 14.9, 14.8, 14.7... to zero)



One way to go at this is from the other direction. When a fire is placed, there's no way I can pre-fill every possible coordinate that the fire could heat. But maybe if I simplify it...

Look at the world as several layers of grids. The largest would have few cells (12x12?). The smallest would have a huge number, and get down to very local info. Have limited number of layers (5?10?)

When a fire is placed, is would be sized to the largest possible layer (maybe if it contains a % of that layer. If no %, then go the smallest). It would ONLY be placed on one grid at a time.

When a temperature calc is requested, the system would find which grid the user is on for each layer, and sum the temps in those grids (so 5 layers would return a max of 5 cells to calc - for example: world wide, regional, state, city, and local. If no value had been placed in the local cell, then it wouldn't get a value for that cell (or it would be zero).

So ANY fire placed, no matter how large or small, would have a calc on a small set of cells. In the DB, only those cells that had been effected would actually have a value (so in the above, layer 1 would have 12 cells populated, layer 2 would have 121, and layer 3 would have 63.

Fires would normally be placed rarely, while requests for temps would be done requested very often - so this might work. The down side is that there's not a easy way to decrease the temp of a single fire over time (which I was planning on doing). Might be able to force a periodic refresh, but that would be annoying.

Above is for biggest fire, which would be world wide - The center is in on two of those big cells, so effect is highest there. Surrounding would be less.


Above is for a smaller fire - it's on a mid-level layer.




The last is on a small-level layer, with only a very local effect.

Didn't show it here, but I think as it goes down, it would effect more of the (smaller) cells.

Spin-off

Working on a spin off, kind of. Based on a idea I had when first doing AE the time, plus some MMORPG I've been mucking with since then.

Trying to brainstorm some ideas (thanks Mike!)

I'm doing something new, and am at a bit of loss on how to implement. Or even design it. I want to do something that swaps Great Circle around.

I want to have any number of points, and have those points have a radius of effect. Then, given a point (A), I want to find all other points whose radius of effect encompasses A.

So you can think of it as a bunch of fires that give off a certain amount of heat (the radius of effect). So given A, I need to find all fires around A that have some kind of effect on A – and in this analogy, what I really want to do is find out the temperature at point A. Some of the fires would have no effect (too far away), yet ones VERY far away may have an effect, if they are really big fires. So a small fire 500 feet away would have no effect. A small one 1 feet away would have a big effect. A really really massive one in Seattle would have a small effect on someone in Columbia.

And I need to do this on a sphere, be able to handle “fires” anywhere on the sphere, handle a large set of fires (many millions), and need to be able to do it fast in the DB.

I’m not sure even sure if I’m thinking about it correctly – maybe it should be some kind of heat map that’s laid on top of a sphere, and the map is changed when a fire starts, so given A I can just get that areas temp from the heat map. But on the other end of it, I can’t pre-calc every possible point on the sphere that the new fire effects.

AE mothballed

So it's really been mothballed since 2003. I think the idea was cool, but the hardware wasn't there. Kinda wish I had worked more on it, as I would have been waaaaay ahead of the location-aware pack that's now around.

The site was moved a while ago to the new dedicated server, and the SQL wasn't working. Spent some time tonight to get it going again, with a note the site is basically a archive.

SR vs AE

Have not written in this journal for a long time. StationRipper started making some money as of 2.0 (released start of this year), so I'm kind chasing after that (getting the word out, improving it, etc.). No idea when I'll get back to AE, as there really hasn't been a lot of movement on getting the average person walking around with GPS's and always-on internet.

Break

So I've got the basics pretty much working. Just need to add apps. The prob is, PocketPC is just kinda sucking. And even if I get some of the apps I've been thinking about going, I don't see a lot of users having always-on internet AND a GPS connection. At least any time soon. I've got to believe at some point this is going to happen, I just don't know when. I'd guess at least three years, but who knows.


I think I'm gonna go take a break for a while - been using streamripper32 for a while now. Like it, but it really sucks, so am thinkin about doing something about that.

Columbia

I've gone downtown Columbia a couple of times with the laptop - annotated around the state house, USC, and downtown. When I walked back around those locations, it popped up the info and a pic (like, what statue on the state grounds I was standing next to). VERY VERY COOL!!!! Only downside was no wireless, so the laptop had the web server running in it. Guards gave me some funny looks, but didn't bother me.

Working

Just saw the Matrix: Reloaded, again. Very cool movie. Picked up a lot of stuff the second time around. I’ll be very interested in seeing what the next is like (and what’s REALLY going on, hehehe). I’m betting on the whole “Matrix in a Matrix” thing.

I haven’t been writing in this much, or working on AE as much as I’d like, but I HAVE been making progress. I actually put the first PocketPC version of LocationAlert up on SourceForge this afternoon. It doesn’t remember stuff (login, selected group, settings) and you can’t Annotate a location yet – but it works fully beyond that. I ended up writing a very neat Worker Thread that does the actually requests to the Web Service. Basically you post a message to the thread; it does its thing, and then posts a message back to you. Works VERY nice. It’s a huge improvement over the VB one, as the interface doesn’t generally loc up (only exception to that is the settings page, which I may nor may not deal with)

I’ve actually created a generic library, which that threading is part of, which I’m going to release as a separate project. It could be the basis for any MFC program to interact with AE. I then may write a COM wrapper, so it can be used from any program, which would be even nicer. It’s on PPC right now, but as far as I know I should be able to move it straight to Windows

I plan on finishing up LocationAlertCE now (all I really have left is to remember stuff, and be able to annotate a location). Then an installer and it should be set. I’ll probably then move to working on the off-line version (which may or may not be the same program). It will only be able to Annotate right now, but that’s OK, as I really NEED something like that to Annotated the area, at least where I can’t get a wireless network. Later on I MIGHT let it download and work offline (but I have a feeling memory on the PPC will rear its ugly head if I do that)

After that, I think I’m going to do “GeoNotes”, which will be a much more bare-bones program. Show the closest location on a single page, be able to flip through pages (closest to farthest), and Annotated (“note”) a location. This will be the first non-techie AE-enabled program that I’ve done. The LocationAlert programs are what I really consider to be the reference software. That is, the thing that tests all the basic features of AE, and shows how to do everything. GeoNotes will be the first attempt at a more mass-appeal application of the technology.

On another note, I’ve continued to think of ways to make money at this. It occurred to me that getting “Complete” PocketPC (or Pilot) hardware together will be a bit complex. The ultimate system would have a WiFi card, GPS, AND a Camera. The software would use all three to Annotated locations (so basically press a button, add a note and upload the lat/long, note, and pic to the main site). REALLY nice, if I can get to that point.

Sooo… If that’s even possible to do, I could sell custom (for now) built PocketPC that have all of that. And maybe the software will be JUST for custom PC’s (or the first that I sell, if you buy it alone). Order all the stuff, put it together, ship it out, make a hundred (???) or so on each. Not something I want to do full-time, but maybe a way to make cash to cover my ever rising experiences, and something I might be able to spin off into a company later on.

Speaking of companies, one of the big things I’ve got left to do is incorporating. Not something I’m looking forward to, as I’ll have to deal with the accounting/tax/etc.

But, I am coming out of the base development now (Wahoo!). It’s just about time to start handling more the business side (form the company, try to find partners, maybe get more serious about finding an investor). My primary concern is how much more it’s going to cost. I’ve already been pumping my spare (and not so spare, sometimes) money into this. I wish I had a pile of it to do all the things that need to get done (like… hire people, hah). I think I’ll have enough money next month to finance the incorporation, at least.

PocketPC

Still working hard on AE, I just haven’t had much time in the last few weeks. I’ve been steadily working on making the web site working better (nailed voting tonight, works much better now).

Most of my time has been spent working on the PocketPC application. I’ve almost finished a wrapper around the HTTP Calls/XML return (pass some params into a function, get a CList or Map back that has the return data). It should make it much easier for someone to develop new stuff. I’m using it for app that logs in, allows the user to type lat/lon and distance, and returns those locs for group 1. The big pieces to finish are the COM port/GPS device stuff, and configuration.

I originally was going to just write it in eVB, but after using the environment for a few hours, I gave up in disgust. VB 6 is bad enough, but eVB sucks sooo very bad. So, I’m doing it in embedded Visual C++, and actually using MFC. Kinda nice, in that I did MFC for so long, it’s pretty easy to write GUI apps. But it’s still a pain, in the API set available on the PocketPC is reduced, and they did some really STUPID things. Like, normally you do a SelectNode with a param, and it returns a result. So WHY on the PocketPC is the result passes as a param? I’m not complaining about the style of doing return codes that way, but now instead of just copying code I already have done, I have to re-write everything  yuck.

I saw NY City is putting in a big public WiFi network next month. I’m going to investigate it a bit, and if it looks like I can just use it, I may plan a long weekend trip to NY soon, to set up a guide. I need to get the Pocket PC app fully working, as it’s a pain dragging around the laptop.

Slashdot

Well, something somewhat interesting. I just noticed a article I posted on Slashdot was accepted. It’s not ABOUT AE (damn! Hehehe), but it’s about GPS and the upcoming war. The interesting thing is, that means a LINK to AE will be on the main Slashdot page, as the URL you get when you click on my name (ward99). Woot! Hehe, should see some big traffic from that, based on the last time it happened.

Location Alert

Wowza. I took LocationAlert and added the ability to Annotated locations. So I may not write LocationAnnotation at this point, as LocationAlert does it. It’s VERY cool. Simple to use, and does exactly what I want. I guess I’m ready to do the PocketPC 

On that note, I installed the SDK this weekend and wrote a simple app, but as soon as I ran it blue-screened XP. Fun. It was trying to access the PDA hooked up to the machine, and, as I haven’t bought one yet, didn’t like it I guess. Guess it’s time to spend yet another $600 bucks *shudder*. I’m tyring not to think how much cash has been poured down AE at this point. If you start calculating my time in, the total cost is starting to get a bit high (and the personal, both in$$$ and personal time work on it)

I’m actually thinking of taking a break soon, I hope after I have something on the PocketPC. I’m starting to get burned out, and my day job is suffering. To this point, I’ve been able to do everything on weekends or (sometimes) weeknights, without the day job getting hit… but last week I was so fried from trying to do both for the last few weeks, that I was kinda flaky, and didn’t get much work done.

Oh… http://sh1.antville.org/ had a post about AE (specifically, the IraqiList) so I’ve been getting a fair amount of traffic, at least a few hits a hour. And several people have used IraqiAlert AND LocationAlert. Nifty! 

I want to try harping IraqiAlert, as I think it has the potential for some press. As the war is about to start, I’m running out of time. I’m going to research Iraqi Troop movements, troops that might have bio/chem. Weapons (as was rumored today). That would be a good addition.

Users...

Well, I kinda cranked on a bunch of stuff tonight, the biggest being the release of LocationAlert. It’s basically a generic version of IraqiAlert. The user can login in as different users (or guest), and choose what group they want to be monitoring. And it returns a list of locations instead of just one (within a user-selectable radius). The user can configure settings (like com port and baud, finally) and it saves/loads settings from the registry. It actually looks pretty damn good, considering I basically ripped the code from IraqiAlert and just made it better. I need to do some testing when my laptop gets back, but it seems to work REALLY well.

The next app will be the location submitter – ah, um, “Location Annotation”? Maybe. I don’t know, I’ll have to think about that a bit.

The first version will only allow the user to take a note for the current location. Once they’ve done that, it goes up to the server, and then they’ll have to edit from there if they want to do anything to it. I really need this app, to start mapping out things. Once the two of these (location alert and location annotation) are working well, then I’ll be ready to move to PDA’s. But I need to use these two to really test out the technology.

I’ve kinda given up on finding wireless in Columbia. I may try to track down someone at USC and see if they’re interested in doing some kind of R&D partnership (doubtful, I’d guess, knowing USC). I’ve thought about just putting the laptop and a WiFi router in my car, and trying to have basically a mobile WiFi node. Park it downtown Columbia, enter a bunch of locations, test. Move the car, repeat. Do something on the USC campus. The only problem is I’m not willing to leave my car on for long periods of time, and I don’t want to run fully off of the battery for the WiFi. If I can figure out a way to just run the 12 volts or whatever off the router, off the car, then... Maybe. We’ll have to see. If I can do that, then I can load the AE server onto the laptop, and configure the software to point there. Kinda cheesy, but hey, It’s Columbia *sigh*

Oh yeah, I’ve posted info about AE on several web forums that deal with location-aware, or GIS or GPS devices. Getting a few hits, and some interested. One thing I really need to do is get some more users 

I’ve been adding a few new locations, which has helped test the system as a normal user would see it. I’d like to make voting better (more visible), and be able to find out who submitted a location (and get a list of locations THEY submitted). That would be nice, as it would allow me to do a top 10 submitter/quality list. Which can help spur people to submit MORE!

StationRipper

So StationRipper kinda took off. Gave it to a friend or two, and they gave to some friends. Next thing I know, my ISP is telling me I'm maxing out my bandwidth. LOL.

Still not a lot of movement on getting people devices with GPS's and internet connections (walking around, at least). It just seems to make SENSE. Wish I was a hardware guy, as I bet there's some cash to be made with that combo.

So I'll probably not be working on AE for a bit. Going to focus on StationRipper, which is a lot of fun and getting a LOT of users!

Fark

I did some work on the web site today. Locations can be deleted. Fixed a Deg-Dec to DMS calc bug, and added the ability to enter locations in MinDec.

I also added some locations, and deleted some (hehehe, nice to have the new delete function).

Tried to submit something to Fark – the article got rejected (no surprise there *sigh*), but did get some traffic, mostly from total fark I’d imagine.

Sourceforge

Well, just submitted a request to SourceForge to add a project for AE clients. I’m NOT planning on open-sourcing the web site at this point, so I sure hope they’re cool with open-source clients. My plan is for all clients that I write will be open source, and available on both the main site and SourceForge. The whole goal here is to get other people writing cool applications, stuff I may even be thinking of. I’d love for someone to take IraqiAlert and function to it, or some up with some other new application. If I can achieve that, then the second stage can really be considered a success. Yeah, lots of users will be part of that success, but if I’m going to move forward, without getting and kind of investment cash, I’m going to need to rely heavily on other developers.

I can see this is already going to drain me dry – already spent $3K+ on hardware, hosting fees will just get to be more, and I’m must starting.

I pretty much have finished IraqiAlert! The web pages are done, the software works as well as it can (with me not having my laptop), the source code is all commented and pretty and I’ve uploaded that source in a zip to the main site (Woot!). I also did the various license pages and such, lots of fun. I need to really start looking around for a lawyer – going to need a corp, and need help with the legal stuff.

Released IA

I released IraqiAlert, or at least put it up on the site. I’m debating whether to try to a fark ad (“Going to Iraqi to help out Saddam? Make sure you know what area’s he’ll most appreciate you protecting!”). It would be nice to get hit a bit, I’m just sure getting fark’ed is the way to go :P

Tomorrow I’m going to take IraqiAlert, comment it, and then take that source and create a generic alert app. Basically get any group the user can access, and let them select which they are monitoring (more than one maybe?). That should be really quick, as IraqiAlert is doing 99% of what it needs to do, just needs to be more generic. Oh, and login I guess 

Then the next will be the “Enter” app, which will allow the user to enter applications. Then the PocketPC stuff. Hehe, just a bit more to go.

I did a bit of a load test tonight. Ran WebWacker against the domain, getting the Iraqi group list (all the locs, ~25K a pop). Ran it for 10 minutes, 5 wackers. It was able to hit it 5265 times (11 a second). I was also using the site, trying to do various things, while that was going on, and I couldn’t tell the difference (e.g., that (smallish) load wasn’t noticeable). I’m going to keep trying various things over the next week, to try to get a feel for what it can take. I was happy to see it didn’t hurt it much, under that load (way to go Active Host!).

I did notice some spikes. On just the AE Service, the base-line for running that transation is 16 milliseconds (that’s from the time the request first comes in to just before the response time is sent. Includes all DB access, login, access validation, XML creation). I see one instance where it took 1150 milli’s, and a handful at 250-400. That’s cutting out any kind of network latency, so it’s probably either the web server being busy or the database (as in… busy doing something other than AE). While Active Host will be able to handle the site for a while, if it takes off at all I’m either going to have to host myself (which I prefer) or get a (at least) better semi-dedicated server. The problem is, of course, I don’t have control over the equipment, so tracking problems is going to be tough.



Oh, and I also added “system” name to everything. Was a real pain in the ass, but I can see that as more software gets written, I’m going to want it (to tell where requests are coming from).

Laptop dead

Well, my brand new vpr Matrix died. I like it a lot, but it’s going to be ~2 weeks before I get it back, repaired. So I’m screwed trying to get things working around the yard, which is pissing me off to no end. I’m really close on the Iraqi alert program. Just needed to do some final testing and more options (like distance to look at, etc.) Don’t know what I’m going to do now. I was hoping to release it before the whole Iraqi war started, or right about that time (which looks to be now, or next week).

I may just release it, but not publish the fast as much as I’d like. *sigh* Oh well, thank you Best Buy :P

I’ve been researching PocketPC’s. I 99% sure that’s the next target platform. Lugging that PC around has been a pain in the ass, so I’d love to get it working. Only problem is, I can’t find a wireless network in Columbia. I’ve considering trying to use the lap top – get a Linksys DSL router w/ wireless, put the web server on the lap top, drive down town or wherever, power it off the car, and at least a temp wireless network going. A pain, but it might be worth it.

The actual location-aware stuff seems to work really well, I just don’t have the infrastructure to really test it out. Once I’ve got a few things working, I may take a trip to someplace that provides a really solid public network. Go to NY or the SF and map out part of the city. Basically get a guide book or history book and walk around, entering facts.

A new idea:

I could develop a War Driving application. People are already using wireless cards and a GPS to map nodes. I could use a AE to combine those two. Start a new group, let people enter info in (actually, Hehe, could just automatically generate the info when it detects a LAN it hasn’t seen before. Basically drive around, hit a new name. See if it’s on AE. If not, send it in). Then you could set in a mode that as you drive around, it tells you how far away the nearest point is. Could be a very cool demo of the technology, and I’d think people War Driving would be a good target audience.