Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a Windows Server 2008 R2 cluster, running a 3rd party trained algorithm that generates geometry and texture maps at high speeds. This is part of a WebAPI, with clients generating new geometry and texture maps on demand. About 1% of the time, when retrieving the largest texture map (a bit over a 1M in size), logic that knows the file size of this file compares the just-loaded-file size with the expected size, and the entire file is not completely there on disk yet.

In the WebAPI logic, I try rereading the file when detecting the entire thing is not being loaded, but the results are the same. However, a few hundred milliseconds later the entire file is there.

I don't want to sleep or delay returning my data when detecting the file is not completely on disk yet. I'd like to somehow force it to be written to disk. Because it's a closed source 3rd party tool writing the file, I can not go and add fflush() or the C++ equivalent to that code.

Also, I'm not running IIS. I'm running a secure WAMP variant called the UniServer. The closed source 3rd party tool is inside a DLL controlled by a C++ program I wrote, which is in turn controlled by a PHP web framework (Drupal) with custom WebAPI logic I also wrote. I include this information so you understand that this Windows Server is configured to operate as Linux/Unix like as possible.

share|improve this question
    
The only thing I can think of off-hand, and I know my terminology is off, is to attach a 'monitor' to the file handle (should be able to look it up by file name) and watch for the close event, which should signify that the file is completely written. The problem is not that the file isn't on disk, it's that the asynchronous write call has not sent all the data yet. Windows should preserve cache coherency, so even if the file isn't completely written to disk another process reading the file should get the correct results. –  Chris S Dec 15 '12 at 3:55
1  
@Chris, I know it may sound odd and unintuitive but Windows can in fact close a file without it having been fully written out yet (but continues to write in the background, even though the file is already marked as closed). To the best of my knowledge this is not documented but has been observed to happen. –  John Gardeniers Dec 15 '12 at 5:43
    
@Blake: "I don't want to delay returning my data when detecting the file is not completely on disk yet. I'd like to somehow force it to be written to disk." Why? Don't the consumers need to wait for the data to be fully available? Why configure a web service to present a file if it isn't fully available? Even if Windows or the storage driver did have a way to compensate for this by only flushing that file, it seems like it would be more optimal to not present the file as available if it really isn't. –  Greg Askew Dec 15 '12 at 14:32
    
@Greg: Well, of course I need to wait or do something to insure I deliver the entire file to the client. I guess I could sleep for a bit, but it sure would be nice to just flush the disk caches... The odd thing here is I have another portion of the same API which zips all the files that resource's assorted GETs deliver (8 files) into a single 10 meg file. That one delivers without issue... but it maybe because I create that zip with PHP in the webAPI framework, not with external C++ processes. –  Blake Senftner Dec 15 '12 at 16:00
    
@Blake: What about creating the file in a staging directory, then moving it to the final destination when it is finished writing? –  Greg Askew Dec 15 '12 at 17:35
show 5 more comments

1 Answer 1

There is a sync program from SysInternals, maybe you could put together something that would incorporate this utility program.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.