Dealing with corrupt stores

Jun 7, 2010 at 7:38 PM

So far, I've encountered a couple different flaky types of corrupt things in the stores I've tested ... I'm using the PSTSDK_VALIDATION_LEVEL_WEAK flag in my code, just to clarify.

First, in table.h, line 660, I changed read_raw_row this way to avoid an unhandled exception:

+        ulong pos = row * cb_per_row() + offset;
+
+        if (pos + sizeof(Val) >= m_vec_rowarray.size())
+          throw std::out_of_range("attempt to read past m_vec_rowarray.size()");
+
*        memcpy(&val, &m_vec_rowarray[ pos ], sizeof(Val));

Also, the next thing I've encountered a lot of is bad named prop_id lookups.  I've had to not only check is_string() == false, (which it is), but also check to make sure HIWORD(get_id()) is zero before considering whether or not to use the named prop further.  I wonder if some validation can be baked into the named_prop class, or maybe the lookup code itself to maybe throw an exception if the named prop fails to pass muster? 

 

Coordinator
Jun 7, 2010 at 7:56 PM
Edited Jun 7, 2010 at 7:58 PM

I'm concerned about why this validation code is necessary (ie, there may be a bug in the sdk higher in the callstack). After all, the row number was validated earlier in this function. This implies the only way to overflow is that the offset they're trying to read is greater than the row length - I wonder who is requested a read at such an offset. It could just be a corrupt store - try with FULL validation to see if a CRC error gets thrown earlier.

Can you give more details about what you mean about a bad named prop lookup? You're finding ids with a non-zero hiword? I don't recall any requirement about the hiword being zero....

Jun 7, 2010 at 8:17 PM

I'll trap the out of range errors and give you a callstack.  One of the problems I face is that I need to read as much of the PST as possible, and not abort for every little corrupt thing I may find (like a CRC error).  Some of these PST's I'm attempting to read have been recovered from a bad drive, or have some other innate corruption that prevents SCANPST from completing or crashes SCANPST entirely.

As for the named prop ... the guid was {cccccccc-cccc-cccc-cccc-cccccccccccc}, it was KIND_MNID, and the ID was cccccccc

I've never seen a named prop of type KIND_MNID with a value greater than a USHORT, so the fact that those can exist is news to me!

 

 

Coordinator
Jun 7, 2010 at 9:23 PM

Ok. The all 0xcccc named_prop doesn't sound kosher either - that sounds like you're looking at an uninitialized variable. Are you sure those values came from disk?

Jun 7, 2010 at 9:49 PM

They appear to have come from disk, but I can't be sure.  The code to fetch the named prop is pretty straightforward (edited for brevity):

  if (propid > 0x7fff)
  {
    named_prop nprop = store->lookup_name_prop(propid);
    // do something with it ... at this point it contains the 0xcccccccc values, and is_string() is false
  }

Coordinator
Jun 7, 2010 at 9:56 PM

Where did propid come from in this example (and what is it)?

Remember, anyone can create a named prop with an arbitrary id and guid through MAPI - it is possible that someone just decided to create a name prop with a guid of {cccccccc-cccc-cccc-cccc-cccccccccccc} and ID cccccccc I suppose.

Jun 7, 2010 at 10:51 PM

I'll have to look.  I was enumerating through the props on a message I think.  Basically, I am converting a pstsdk::message object to an IMessage object, which of course involves walking every prop, converting it to an LPSPropValue, and writing the prop to the IMessage.  When I come across a named prop range, I query for the named prop, and re-create the named prop in the IMessage when copying it over.  Rinse and repeat for recipients, attachments, etc.  Everything's working quite nicely, but occasionally I find a rogue prop or funky named propid.

It's a nice way to verify the PST SDK, actually ... Accounting for every possible property, copying every field, embedded object, etc -- walking through every sub object and so forth.

Coordinator
Jun 7, 2010 at 10:54 PM

Ok. Try running the nameprop sample on the PST in question; see if the weird named prop shows up in the output there. Other than that, I can't do much else without the PST - but maybe if you can debug into the process of reading the name prop, looking at locals along the way, etc, maybe you can get an idea as to what is going on. Same with the table issue.

And also; I'd recommend you try it with full validation on. If you are hitting CRC errors that are ignored under weak validation, that would explain things and make this a lot less interesting.

Let me know if you find anything out, thanks again!

Jun 8, 2010 at 6:11 AM

With validation set to FULL, I get crc_fail errors when calling lookup_name_prop(), which is actually quite fine.

Unfortunately there's a host of other issues with FULL ... specifically, it's too chatty on a corrupt PST.  I end up not being able to open many PST's, and have trouble enumerating through them with FULL.  It's just too many throws.  Messages that I could mostly read, I now can't read at all ... property bags that had partial information now throw errors and I get nothing.  I kindof want the accountability of FULL, with the recoverability of WEAK :)

 

Jun 8, 2010 at 4:22 PM

What might be a cool option is if the validation level could be set at runtime rather than compile time.  I could see a lot of practical uses for that, especially if it could be adjusted upwards right before a function call, and then adjusted back down afterwards ... giving you some finer grained control over just what throws an error.