I finished the robot last night to do the checking but I am going to wait until I have enough links in the home_page table before I start running the checker. I have been using vmstat and iostat to monitor swap my system and it is not being taxed in the slightest so I am going to start using more than one robot.
1.1M links_found
1.2M home_page
17K downloaded and parsed