Defining some terms might be in order.
A specular map determines where the bright reflective highlights on your object occur, as well as what color they are. Metallic objects have highlights that are the same color as the object. Most other substances have white highlights. The specular pass is usually added or screened onto the diffuse pass, so in order to achieve white highlights, you must usually make a specular map that has colors that are complementary to your diffuse (color) map. In Photoshop, you should invert the image (Ctrl+I) to get your starting point. From there, modify the image such that areas of the texture that should be matte (non-reflective) are very dark, and areas that should be shiny are bright.
A normal map instructs the renderer to create simulated lighting on a surface and allows you to add apparent detail without additional geometry. It's similar to a bump map, but instead of just indicating variations in surface height, it actually dictates the angle of a pixel with respect to the geometry's normal (a normal being a line perpendicular to the surface). Not many people paint normal maps by hand or in Photoshop. They are usually created with 3d sculpting software such as ZBrush or Mudbox. I think that Modo has a sculpting mode built-in, but I don't know how it works.
Anyway, as regards your corrupted file, you've now learned one of the great important lessons in 3d: File versioning. Save new versions frequently, so you can backtrack if an approach isn't working or save yourself if something gets corrupted. I usually have 30 - 50 scene files by the time I'm done modeling something in Maya. If you need to save disk space, you can delete the excess at the end once you're satisfied with your results.
edit:
Any chance you might share the geometry itself in .obj format? Does the trial version of Modo even allow that?
Btw, if you're not opposed to trying a different piece of software, SideFX offers a free learning edition of Houdini that watermarks the renders, and a non-commercial but non-watermarked version for only US$100. Houdini is often used in the film vfx industry for its superb dynamics and simulation capabilities, but it's also a solid modeling and rendering package. I've been learning it slowly, myself, although I still use Maya for my projects.
Reply With Quote