I'm dealing with an issue and have been unable to find any posts or documentation that already address this specific problem.
This issue is regarding a heavily accessed web site, written in .NET 3.5 and running on IIS7. The application utilizes a Lucene search index via UNC path set within the application's Web.config. This UNC path is the same on each of 16 web servers and they all access the same index in this single location. The application runs in classic mode (vs integrated) and it uses an identity and impersonation. The account has sufficient permissions. The web servers are all currently Windows 2008 servers and the source file share is also Windows 2008 server running SQL Server 2008 as well. The servers are always in a Workstation configuration, never a domain and no AD. All servers have one main user account configured identically with identical permissions and password.
This setup has been working fine for years under a few configurations. Recently with web servers running Windows 2003 and IIS6 accessing this remote UNC file share on a Windows 2003 server running SQL Server 2005. And most recently with web servers running Windows 2008 and IIS7 accessing this remote UNC file share on the same Windows 2003 with SQL as just referred to. The problem occurs only now after having moved this UNC share to this new Windows 2008 server with SQL.
The site runs well currently and each web server has access to the index and successfully utilizes the index as it should. However, periodically (between 1 and 15 minutes based on activity and a timed service) the Lucene "segments" file is removed and replaced (it's changed). The current file, e.g. "segments_xyz" is replaced with "segments_zyx". Not every time this occurs, but most times, the application seeks out the previous file, not the current file. This results in a FileNotFoundException and the .NET error reported is: System.IO.FileNotFoundException: Could not find file '\X.X.X.X\Index\20101201\guid-x-x-x-x\segments_zyx'. This lasts for 1 to 3 seconds and it occurs on all of the servers. This is reproducible in that I can access the site directly on one server, watch for the file to change and when it does I can refresh the page and receive the error for 1 to 3 seconds.
Some points and theories:
I've suspected this is a permissions issue though that doesn't seem likely. I've exhausted permission configuration options. I've gone so far as to configure the file share with complete public access utilizing the Everyone user account group and the identity account has been given full administrator privileges by being included in the Administrators group. I've adjusted the Authentication ASP.NET impersonation settings among other things.
Suspecting some kind of UNC file share caching I've reviewed the Shadow Copy (VSS) settings on the source share server and I've looked at each web server to ensure there are no previous versions, etc. This also seems unlikely a cause to me.
Running the application in Integrated mode is not an option
This issue occurs on every single one of our 16 web servers and it appears they all (or most) suffer at the same time through the same file change iterations.
To be very clear, this issue only showed itself when I moved the share from our Windows 2003 w/ SQL server to our Windows 2008 w/ SQL server. Configurations are as identical as possible and there is nothing special about the configuration, it's a simple standard file share with common permissions configured.
I've completed the requisite reboots, reconfigurations, service resets, etc.
I've attempted to change the UNC path configuration to a mapped drive configuration but there are application issues that prevent that option currently. I do not know if that would solve this current problem and I'd much prefer not to implement this as a solution.
This issue causes 1000's of errors per day. Hopefully someone has some ideas? I'll be eternally grateful for any help on this!