Rick's World

Damn, these spammers are fast!

Recently, I upgraded my site to the newest version of the blogging software. Unfortunately, in doing so, I lost all my changes that removed the displaying of the Referer links. This means that the referer links are back up on my site. The spammers didn't take long to notice! I just got done deleting over 1,000 referer spam hits from just the last DAY! I also noticed recently that my changes to remove the referer links missed a spot. There was one spot left where the the referer links were still being displayed. It wasn't on the main page, which is why I missed it.

Another interesting thing I noticed is that last month, the top browser used to access my site (49% of all requests) was a browser named 'Wget'. No, it wasn't Internet Explorer or Mozilla/Firefox as you'd expect. What is 'Wget'? It's not really a browser at all. It's actually a very basic command line program, that's very common on Linux that is used to recursively download an entire web site. This means that the spammers are likely using a Linux/Unix computer of some kind to run some kind of script that uses WGet to download the pages of all the different blogs they spam, automatically. This has the side effect of sending their spam refererer links to the website they are downloading, hence their links get automatically added to the site. I wonder if I can ban the WGet browser completely from my site? Does my web host software support this or am I going to have to do some custom hacks to the blog software? Looks like I have some programming to do this weekend.

One of these days I need to write a nice long blog describing some of my referer spam findings and how to fight these guys...

Working in the computer field is a challenge. You are constantly chasing a moving target. In most jobs, once you learn the basics of your job, you can pretty much do the same thing every day until you retire. Not with computers. The field of computers changes so fast that you have to constantly try to keep up with the latest developments. If you don't keep up, then the job you are doing today could very easily dissapear and you will find yourself without a job. I often tell anyone who is considering the computer field for a career "If you are afraid to learn new things, don't get into computers. You are going to be learning something new every day." People who prefer the same old routine at work every day for the rest of their lives are bound to be dissapointed. You have to constantly be able to adapt to the changing landscape. There will always be something you don't know how to do, but will have to do anyway. You have to figure out how to do this. This often requires a lot of logic. You have to look at a problem and reason out, step by step, what the problem is and how to fix it. This is not an easy skill to pick up. It requires practice. I don't think you need to be any more 'intelligent' than the next guy to do this, you just need to practice at it. It's a skill, just like any other. The more you practice, the better you become at it. I've been doing this stuff professionally for 12 years now. It comes fairly easy to me now. This is just because I've had a lot of practice at it and I know what mistakes to avoid. It doesn't make me any smarter than the next guy, just more experienced.

When I first graduated college, and for the first few years after that, I constantly worked in a state of panic. There was so much I needed to know to do my job, but I didn't know it! My God, if I don't learn this stuff, I could lose my job! These days, my point of view has mellowed to the point where I don't worry about it. I know enough to do my job. If I need to know something else, I can learn it. That's why I'm not afraid of tackling new, unfamiliar things - such as the guitar. I know that as time passes, I will get better. You just have to get past that point at the beginning where you feel overwhelmed by all the stuff you have to learn. I'm beginning to think I'm entering this stage again at work.

Lately, it's become clear to me that my skills in SQL are sadly lacking what they should be. Every job that I've had for the last 6-7 years has required the use of SQL - some of it very complicated SQL. Considering how long I've been doing this stuff, my knowledge in the area is very basic. I have come to the conclusion that I need to greatly improve my abilities in this area. I think I need to spend a lot of my spare time over the next few weeks/months brushing up on my SQL skills. I'm trying to decide the best way to approach this.

Back when I was first learning .NET I got it into my head that I would need to earn a Microsoft certification in the subject, in order to prove on future jobs that I could do the work. The first certification you can earn in .NET requires the passing of three certification tests. I passed the first two. I never got around to taking the third test and getting that certification. The reason I never finished that last test was that around the time I was studying for it, I found a new job that did not require .NET. I was back to doing what I know and what I love - C++. I figured, why bother with the certification if it is not something I'm going to use in my job anyway.

I had always had a low opinion of certification tests in the past. Why the heck should I have to take a test to prove I know how to do something? Just give me the work and I'll do it. I can figure it out! When my circumstances at the time forced me to reconsider this outlook, I began to realize just how useful these certification tests are. While studying for these tests, you are forced to learn a lot about the subject. If you are going to be tested on the subject (at a cost of >$100 per test), you are going to make damn sure that you don't waste that $100 you spent on the test. I could very well have just started doing the work and muddled along, and things would get done. The problem is, I would not have been doing them well. In other words, I may have done the job, but I would be doing it poorly. There is a right way and a wrong way to do things. I was doing it the wrong way. By taking the tests, you learn the right way to do things, thereby becoming a 'master' at things. I think the test it self is largely unimportant. It's the effort of studying for it that is important. In retrospect, I'm glad I took those two tests. They made me a MUCH better .NET programmer than I would have been had I continued to muddle through. I've been at my new job now for 9 months. In those 9 months I've largely ignored .NET. I've often considered going back and taking that last test and getting that certification so at least I can say that I finished what I started.

So how does my need to learn SQL and my past experiences with the .NET certification tests relate? Here is how. I've been seriously considering pursuing an MSDBA certification (DBA is short for Database Administrator). DBAs are generally known as experts at databases. By pursuing this certification I can solve my problem with my lack of good SQL knowledge. There is some overlap when it comes to the certification tests for MSDBA and the MCSD (the .NET certification). This means that if I take the tests in the right order, I could conceivably acheive the MCSD certification, then the MCDBA certification. This would not only 'finish' my knowledge of .NET, it would solve my SQL knowledge problem, and give me a potential new career path to boot - DBA! I've already got the books I need for this. I'm just not looking forward to the months of studying and the hundreds of dollars worth of tests.

I need to give this some serious thought.

Ok, so I am a developer / programmer / software engineer. Choose a name. It doesn't matter to me as long as the paycheck clears! Some people I know get very bent out of shape over titles.

I am a hardcore fan of C and it's related languages (C++, C#, Java, etc.). I am NOT a fan of the 'Basic' style languages (Visual Basic, Delphi. etc.) Why? I don't know. I guess because C style languages are clean and not too wordy, but at the same time are very readable (unless you are one of these programmers who likes to impress people by writing bizarre code - that's a story for another blog). Today I made my first real foray into programming in Visual Basic (shiver). The main reason I don't like Visual Basic is because it's too wordy and it is not a very well designed language. Things have been just tacked onto it for years. It's a mess. Unfortunately, since the project I work on at work is underfunded, I'm the only developer on the project. There is more work than people. Since I'm the only developer available, I had to make some changes to and old version of the web site that goes with the project. This web site was written in Visual Basic 6, a program that is SEVEN years old, and no longer even supported. The site is a confusing mixture of Visual Basic, ASP, and COM+ controls.

In my attempts to get this confusing and ancient mixture of code working on my machine, I managed to screw something up so badly that now I can't compile *ANY* programs in ANY language! Me and several other coworkers spent the afternoon trying to figure out the bizarre set of problems on my machine. So we ended up resorting to the classic Microsoft solutions of 'reboot' and 'reinstall'. Several reboots didn't help. I uninstalled ALL of the development tools and proceeded to start reinstalling everything in the HOPE that it fixes the problem. I will likely be working on this all day tomorrow as well. If the reinstall doesn't work, I'll likely have to send my laptop off to the IT department for a complete reinstall of Windows! Kiss the rest of my week goodbye.

Compound that with other delays and work and today truly sucked. I *HATE* when crap like this wastes time!

Just noticed some interesting things you can find in the Windows environment variables:

In my previous blog, I covered how C# handles memory management. In this one, I'll explain how Cocoa handles memory management and why I feel it is a better way to do things than the C# way.

In C#, you allocate objects on the heap using the 'new' operator. This creates an object on the heap and returns a 'handle' to the object. This 'handle' to the object is really just a disguised pointer. The C# runtime is still using pointers, just like C++ does. The only difference is that C# 'hides' the fact that it's a pointer from the user. This is one of the reasons I emphasized so strongly in my previous blog about what makes a good programmer, the importance of understanding pointers. C# users are using pointers, they just don't realize it! Pointers are still relevant in C#, even though they don't appear to be there. They actually are. This pointer to the newly allocated object is then passed around (with it's true pointer nature hidden from the programmer) and the programmer uses the object however he wants. Then when the programmer is done with this pointer he simply stops using it somewhere. The C# runtime is responsible for figuring out when the object is not being used anymore. Once it figures out that the object is no longer used, it addes that pointer to a list of memory blocks that can be deleted. Eventually that memory block gets deleted during idle processing. (which can be a long time off in a cpu intensive application). Essentially, the C# runtime uses an internal reference counting system to keep track of the lifetime of the pointer. Whenever another copy of that pointer is used, the reference count is increased. Whenever the copy of that pointer is no longer used, the reference count is decreased. When the reference count reaches 0, the object is no longer being used and it gets put in the list of memory blocks to eventually be freed.

Cocoa does this differently. Cocoa's way is sort of a 'middle ground' between the very low level C++ way of memory management (do everything yourself) and the C# way of memory management (the runtime does everything). In Cocoa, all objects are reference counted as well, but the difference is that YOU are responsible for the reference counting. If you need to save a copy of the pointer, you increase it's reference count by calling the 'retain' method on that object. When you are done with the pointer, you call the 'release' method on that object. Whenever 'release' is called, if the reference count hits 0, the object is deleted immediately. You still create the object using the 'new' method, but you 'delete' it using the 'release' method. Essentially, the reference counting takes care of some of the lifetime issues involved in managing pointers, but ultimately, YOU control when an object is deleted. This makes for a much more efficient program when it comes to memory usage. Yes, the Cocoa way is a bit more work, but I feel it is a small price to pay for efficiency.

My friend Dave, wrote a blog a few days ago, commenting on what he thought made a good programmer. I thought I was respond by listing my own qualifications for a good programmer:

• Knows the basics. Know how to open a file, read/write to it, and close a file. Know how to write functions, know basic memory management. In C++, know the difference between 'delete' and 'delete[]' and when to use each one. If you are a programmer and you can't tell me why the following C code is bad, you don't deserve to be a programmer. char* s = "A"; strcpy( s, "This is a test" );
• Keep up with modern technologies. At least having a passing familiarity with some of the other common languages out there - C++, C, Java, C#, Perl, etc. Have at least a passing knowledge of some of the commonplace technologies used in softwae today. If you don't know what XML is, or what object-oriented programming is, you don't deserve to be a programmer.
• Does NOT write 'clever' code. Noone is going to care if you can use some bizarre language construct to write code. It's not going to impress anyone and its just going to unnecessarily confuse people. The code you write should be simple, to the point and easy to understand.
• Programs with maintenance in mind. Let's face it, the chances are very good that at some point in the future, someone else is going to look at your code. Why make their lives more difficult by writing oddball code (see previous item) or making the code unnecessarily obscure. Write nice, simple, easy to read, well documented code. Why make the next guy's life more difficult than it needs to be? For that matter, why make your own life more difficult than it needs to be? Chances are, 6 months to a year from now, you are going to have to go back and fix a bug in some old code you wrote. You likely won't remember everything you wrote. This means that you'll end up having to figure out your own code! Make your own life easier by making it easy to understand.
• Don't be arrogant. I've encountered many programmers over the years who have a superiority complex. They think they are better than the 'little people' in the company. I used to work in a company where they intentionally made the tech support department afraid of the development department. The head programmer would routinely 'talk down' to the tech support people. This is unnecessary. These people are just trying to do their jobs, just like you are. Don't make their lives any more difficult than they need to be. Everyone in a company is important. You are not any better than them, just because you make more money. Without the tech support department, noone would be using your software, and you'd be out of a job. Without the QA department, your software would be so buggy that noone would use it and you'd be out of a job. Your software provides them with a job, and their support of the customers allows you to keep your job.
• Test your code! Yes, you are a programmer and yes you are a smart person, but no, you are not perfect, despite what you may think. Test every line of code you write. Test every possible path through a function. Try throwing unusual data at a function and see what happens. Investigate organized testing development methodologies like Extreme Programming and put them to use in your every day development. Even if its not a 'company sanctioned' technique, use it and don't tell anyone. It will lead to better, more reliable code.
• Don't be offended when people find bugs in your code. You are human and you make mistakes. Live with it and learn from it. If someone in QA finds a bug in your code, be glad! It's better to fix the bug before it's released to the customers than after. It makes you and the company both look better. QA is there to 'cover your butt'. Let them and be glad they are helping you! Yes, it is a humbling experience when someone finds a bad bug in your code, but you must learn to accept it as a part of life. Use it as a learning experience. What did I do wrong here and how can I prevent myself from doing it in the future? Once you learn to stop being offended by criticism, you learn that this can be used to better your abilities in the future.
• Learn, learn, learn! This goes along with my previous item about keeping up with modern technologies. The computer industry changes VERY rapidly. You have to keep up with the times. Don't tie yourself to one technology too tightly. Always be reading up on new technologies. Try new things. Be open to new ideas. Considering how fast the computer industry changes, if you get too tightly tied to one technology, you could suddenly find yourself out of a job and with no prospects of finding a new one. I once told someone who was considering entering the computer field - "If you are afraid of learning new things, find another career. You are going to be learning something new every day. Every day there's going to be something you don't know how to do. You are going to have to figure it out!"
• Learn object oriented programming. Let's face it, object-oriented programming is here to stay. Learn to think in terms of objects instead of functions. Learn the concept of information hiding. Learn polymorphism. Learn what virtual functions are. Learn what abstract classes are. Learn what interfaces are.
• Learn socket programming. Let's face it, this is a networked world. Everything runs over the Internet these days. If you don't know how to send/receive data over sockets (or even what a socket is), you are in trouble.
• Learn the basics of database programming. Databases are everywhere. In any job you get these days, you will likely be interacting with one. Know what a table is. Know what an index is. Know what database normalization is. Learn basic SQL. Know what a stored procedure is. Know what the INSERT, DELETE, UPDATE and SELECT commands in SQL do.
• Write cross-platform code! Despite what Microsoft may think, there is more than one operating system in existence in the world. Don't write file formats that are locked to one operating system. Don't write network protocols that send/recieve binary data. Only send/receive text data. Every major Internet protocol uses text for all it's commands (HTTP, FTP, NNTP, SMTP, IMAP). There's a reason for this. These protocols are designed to work on any computer and any operating system.

The first subject I want to talk about, is what I consider to be the most important subject of all, when it comes to writing software - memory management.

Why is memory management so important? Because I would estimate that 90% of all bugs, crashes and security holes that occur in software are related to memory management. It is a tricky subject to get right. Memory management is the process of allocating memory, using it and freeing it when you are done with it. A well written program uses as little memory as possible. It should not 'hog' the machine. This is one area, where I feel that C# fails, despite admirable efforts to succeed.

In C++, you are completely on your own when it comes to memory management. You tell the program exactly when to allocate the memory and exactly when to free it. This is the simplest and most straight-forward way to do things. It is also, unfortunately, the most error-prone. As programs get larger, and more complicated, you tend to lose track of where memory was allocated, how big it is, and where you need to delete it. This leads to frequent 'memory leaks' where you allocate memory and forget to delete it. This memory is then essentially lost for the remainder of the life of the program. Over time, these memory leaks can add up causing losses of performance or potentially causing the program to crash due to an out of memory condition. Plus, many programmers are unaware of the fact that in C++, there are two different ways to delete memory. You use one technique to delete a single object, and another techique to delete an array of objects. If you use the wrong one, you can potentially leak memory or even cause a crash. While this way of handling memory management is prone to errors, it also gives you great control. If you are able to control exactly when memory is allocated and when it is deleted, you can write your program to use memory very efficiently. This is a good thing.

In C#, things work very differently. In C#, you allocate objects and never delete them! The idea here is that the C# runtime will keep track of when an object is used and when it isn't. When the object is no longer used, it automatically deletes the memory for you. This design was intended to solve the problem of forgetting to delete the memory and having memory leaks all over the program. It has been my experience, however, that this feature has had the opposite effect. It's lead to more of what I would consider 'memory leaks'.

The problem with the way the C# runtime deletes memory is that it doesn't delete the memory immediately once it determines that the memory is no longer being used. This memory is instead placed in a pool of memory to be deleted later, whenever the runtime decides to do it. While this memory is in this pool, it is effectively 'leaked'. It's still wasting memory, even though it's not being used by the application. There is no way to know when this memory will be cleaned up by the runtime and thus no way to know at any given time, exactly how much memory your program is using! If your program is creating many temporary objects, this can lead to unusually high usage of memory by your program. This is wasteful in my opinion.

The other problem with the C# way of deleting memory is that it leads programmers to develop very dangerous habits. If you learned to program on a language like C#, you've developed the habit of just going around happily creating objects and never cleaning up after yourself. This can be a very dangerous habit to have if you suddenly have to switch to a language like C++ or Objective-C where this is your responsibility.

A while back, I bought my first Apple Macintosh - a Mac Mini. If you've been following my blog, I'm sure you've read some of my articles about what I like about the Mac and my occaisional comments about what I *don't* like about Microsoft. Since I am a programmer by trade, naturally, once I got the Mac I was curious to learn how to write programs for it. While studying this, I came across Cocoa. Cocoa is the Mac's implementation of Objective-C. Objective C is a variant of the regular C language that many programmers are familiar with. It basically takes existing C, and adds a thin layer of object oriented features to it. It appears, that Cocoa is the way to go when it comes to writing Mac apps. There are MANY Cocoa classes provided by Apple to make writing Mac apps easier. C#, on the other hand, is a relatively new language, invented my Microsoft. It is a variant of C++ (notice how the '#' sign looks like two '+' signs stuck together?). I always thought that C# (pronounced "C-Sharp" was a very clever name for a C++ variant. Over the last few years, Microsoft has been moving more and more of their development over to C#. C# has rapidly evolved into a very capable language for developing Windows and web applications.

I wanted to write this blog as a way of expressing my thoughts on the pros/cons of each of these languages and which I think is better , from a programming point of view. I expect this will be a long entry, so it could take several blogs to complete. (Yes, Rick is writing another 'book'.)

To give you an idea of where I am coming from in this article, a little background is in order. I have been programming in C++, mostly on Windows, as my day to day job for the last 12 years. I have a bachelor's degree in Computer Science from Florida State. Most of those last 12 years have been using 'plain-vanilla' C++ on Windows, although I did spend a year doing C++ on Linux, about 6 months doing Java, and about 1.5 years doing C# on Windows. My experience in Cocoa has been admittingly brief - only about 1 month of playing around with it in my spare time, but despite that, based on my past experience with object oriented languages, I can easily 'project' Cocoa's features onto my C++ experience and be easily able to see how these things would apply in day-to-day situations.

Now that we've got that out of the way, on to the fun!