System is non production, running server 2008 enterprise (localized in Dutch).
"Bare Metal Recovery" selected, backups scheduled daily using Windows server back-up.
Backup destination is a local disk, with plenty of disk space still free.
backups running fine up and until 6 june.
At that date we installed SQL server 2008 R2 and Advantage ADS (other database server).
I recently noticed that the backups have been failing since 6/6.
Investigation performed so far :
-> error message in backup wizard : "the system writer is not found in the backup"
-> confirmed that system writer is not in the writer list when doing vssadmin list writers in a cmd box.
-> found a gazillion articles on google indicating wrong permissions on %windir%\winsxs\Filemaps\*
reference article : http://support.microsoft.com/kb/2009272
most of these also mention problems with CAPI2 (cryptographic provider) but I see no evidence of that in my application log, only event ID 517 is present, indicating error 2155347997.
I follow the article instructions anyway and update file permissions but system writer is still not listed.
Then the article mentions that Event ID 8213, indicating permission issues, might be showing, but it's NOT.
registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\VssAccessControl is already on 1
-> NO log is created under C:\Windows\Logs\WindowsServerBackup\ during the backup attempt.
-> I was able to restore the winsxs\filemaps folder from the last successfull backup without any problem.
log file was created during that restore.
-> I'm not a big hero with process monitor but quickly comparing outputs to a working machine yields no obvious culprit. It's too verbose to really dig in without advanced knowledge of what you're looking for
-> using vsstrace.exe I get this output repeating over and over.
There are error messages in there but I have no idea what they mean and searching the net doesn't give me anything :
ERROR 1 CONTEXT (repeaded approx 300 times)
[10:14:23.075 P:150C T:1388 INCTYPEH(0256) GEN] ENTER[CVssAutomaticLock2::CVssAutomaticLock2]
[10:14:23.075 P:150C T:1388 INCTYPEH(0197) GEN] ENTER[CVssCriticalSection::Lock]
[10:14:23.075 P:150C T:1388 INCTYPEH(0197) GEN] EXIT[CVssCriticalSection::Lock] Time spent: 00:00:00-0000; total: 0; HRESULT: 0
[10:14:23.075 P:150C T:1388 INCTYPEH(0256) GEN] EXIT[CVssAutomaticLock2::CVssAutomaticLock2] Time spent: 00:00:00-0000; total: 0; HRESULT: 0
[10:14:23.075 P:150C T:1388 CORPRVMC(0363) COORD] ENTER[CVssProviderManager::GetProviderItfArrayInternal]
[10:14:23.075 P:150C T:1388 CORPRVMC(0363) COORD] EXIT[CVssProviderManager::GetProviderItfArrayInternal] Time spent: 00:00:00-0000; total: 0; HRESULT: 0
[10:14:23.075 P:150C T:1388 CORPRVMC(0468) COORD] ENTER[CVssProviderManager::GetProviderInterfaceInternal]
[10:14:23.075 P:150C T:1388 CORPRVMC(0470) COORD] CONTEXT[CVssProviderManager::GetProviderInterfaceInternal] Adding context: 'Aanroepbare interface voor deze provider verkrijgen' (0)
[10:14:23.075 P:150C T:1388 CORPRVMC(0471) COORD] CONTEXT[CVssProviderManager::GetProviderInterfaceInternal] Adding context: 'Provider-id' = '{b5946137-7b9f-4925-af80-51abd60b20d5}' (2)
[10:14:23.076 P:150C T:1388 CORPRVMC(0472) COORD] CONTEXT[CVssProviderManager::GetProviderInterfaceInternal] Adding context: 'Klasse-id' = '{00000000-0000-0000-0000-000000000000}' (2)
[10:14:23.076 P:150C T:1388 CORPRVMC(0473) COORD] CONTEXT[CVssProviderManager::GetProviderInterfaceInternal] Adding context: 'Context van snapshot' = '-1' (2)
[10:14:23.076 P:150C T:1388 CORSOFTC(0155) COORD] ENTER[CVssSoftwareProviderWrapper::AddRef]
[10:14:23.076 P:150C T:1388 CORSOFTC(0156) COORD] Provider Wrapper AddRef(00000000004006D0) 2 --> 3
[10:14:23.076 P:150C T:1388 CORSOFTC(0155) COORD] EXIT[CVssSoftwareProviderWrapper::AddRef] Time spent: 00:00:00-0000; total: 0; HRESULT: 0
[10:14:23.076 P:150C T:1388 CORSOFTC(0163) COORD] ENTER[CVssSoftwareProviderWrapper::Release]
[10:14:23.076 P:150C T:1388 CORSOFTC(0164) COORD] Provider Wrapper Release(00000000004006D0) 3 --> 2
[10:14:23.076 P:150C T:1388 CORSOFTC(0163) COORD] EXIT[CVssSoftwareProviderWrapper::Release] Time spent: 00:00:00-0000; total: 0; HRESULT: 0
[10:14:23.076 P:150C T:1388 CORSOFTC(0123) COORD] ENTER[CVssSoftwareProviderWrapper::QueryInternalInterface]
[10:14:23.076 P:150C T:1388 CORSOFTC(0128) COORD] Error while obtaining an interface interface 0x80004002
[10:14:23.076 P:150C T:1388 CORSOFTC(0123) COORD] EXIT[CVssSoftwareProviderWrapper::QueryInternalInterface] Time spent: 00:00:00-0000; total: 0; HRESULT: 0x80004002
[10:14:23.076 P:150C T:1388 CORSOFTC(0155) COORD] ENTER[CVssSoftwareProviderWrapper::AddRef]
[10:14:23.076 P:150C T:1388 CORSOFTC(0156) COORD] Provider Wrapper AddRef(00000000004006D0) 2 --> 3
[10:14:23.076 P:150C T:1388 CORSOFTC(0155) COORD] EXIT[CVssSoftwareProviderWrapper::AddRef] Time spent: 00:00:00-0000; total: 0; HRESULT: 0
[10:14:23.076 P:150C T:1388 CORPRVMC(0468) COORD] EXIT[CVssProviderManager::GetProviderInterfaceInternal] Time spent: 00:00:00-0000; total: 0; HRESULT: 0x80004002
Error message 1 :
[10:14:23.076 P:150C T:1388 CORSOFTC(0123) COORD] ENTER[CVssSoftwareProviderWrapper::QueryInternalInterface]
[10:14:23.076 P:150C T:1388 CORSOFTC(0128) COORD] Error while obtaining an interface interface 0x80004002
ERROR 2 CONTEXT (repeaded 7 times) :
[10:14:33.503 P:04A0 T:14E4 WRTWRTIC(2274) WRITER] EXIT[CVssWriterImpl::RequestWriterInfoInternal] Time spent: 00:00:00-0015; total: 0xf; HRESULT: 0
[10:14:33.505 P:150C T:1388 WRTASRWC(0183) WRITER] EXIT[CVssAsrWriterBackup::_AddComponents] Time spent: 00:00:00-0015; total: 0xf; HRESULT: 0
[10:14:33.505 P:150C T:1388 WRTASRWC(0627) WRITER] ENTER[CVssAsrWriterBackup::TranslateWriterError]
[10:14:33.505 P:150C T:1388 WRTASRWC(0628) WRITER] TranslateWriterError - 0x00000000
[10:14:33.505 P:150C T:1388 WRTASRWC(0633) WRITER] TranslateWriterError - 0x00000000
[10:14:33.505 P:150C T:1388 WRTASRWC(0627) WRITER] EXIT[CVssAsrWriterBackup::TranslateWriterError] Time spent: 00:00:00-0000; total: 0; HRESULT: 0
[10:14:33.505 P:150C T:1388 WRTASRWC(0146) WRITER] EXIT[CVssAsrWriterBackup::OnIdentify] Time spent: 00:00:00-0015; total: 0xf; HRESULT: 0
-> vssadmin list writers output :
vssadmin 1.1 - Opdrachtregelbeheerprogramma voor Volume Shadow Copy-service
(C) Copyright 2001-2005 Microsoft Corp.
Naam schrijver: 'Task Scheduler Writer'
Schrijver-id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
Id van schrijverinstantie: {1bddd48e-5052-49db-9b07-b96f96727e6b}
Status: [1] Stabiel
Laatste fout: Geen fout
Naam schrijver: 'VSS Metadata Store Writer'
Schrijver-id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
Id van schrijverinstantie: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
Status: [1] Stabiel
Laatste fout: Geen fout
Naam schrijver: 'Performance Counters Writer'
Schrijver-id: {0bada1de-01a9-4625-8278-69e735f39dd2}
Id van schrijverinstantie: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
Status: [1] Stabiel
Laatste fout: Geen fout
Naam schrijver: 'ASR Writer'
Schrijver-id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Id van schrijverinstantie: {7ddb5279-28d6-4f25-af34-535b05c33460}
Status: [1] Stabiel
Laatste fout: Geen fout
Naam schrijver: 'Shadow Copy Optimization Writer'
Schrijver-id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Id van schrijverinstantie: {5c4742bd-a15c-4769-9e11-7422623c4f35}
Status: [1] Stabiel
Laatste fout: Geen fout
Naam schrijver: 'SqlServerWriter'
Schrijver-id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Id van schrijverinstantie: {6b99c086-1f34-4cb8-bea5-887846f86a75}
Status: [1] Stabiel
Laatste fout: Geen fout
Naam schrijver: 'Registry Writer'
Schrijver-id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Id van schrijverinstantie: {7a76da99-a6f1-4b4b-85f4-5040079f6487}
Status: [1] Stabiel
Laatste fout: Geen fout
Naam schrijver: 'COM+ REGDB Writer'
Schrijver-id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Id van schrijverinstantie: {93839243-d67d-49d2-856e-49a1ec4dc87b}
Status: [1] Stabiel
Laatste fout: Geen fout
Naam schrijver: 'WMI Writer'
Schrijver-id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Id van schrijverinstantie: {2ae99e39-7c2c-4fbb-809a-9d42b4aba1ff}
Status: [1] Stabiel
Laatste fout: Geen fout
notice that also the BITS writer is missing, but the others seem just fine ...
-> followed instructions in this article : http://briandrab.spaces.live.com/blog/cns!B69C8C4D664E3F01!252.entry?sa=609120164
moved all files under filemaps to a different folder but that didn't work either so i just restored them.
-> I probably could restore the backup from 6/6 completely and reinstall sql + ads, but I want to find out what is happening before I do that, I find not much info on the web over what mechanism makes the writers visible exactly.
This would be a lot of work with no guarantee that things don't break again.
At this point I welcome any suggestion, but i'd like to see some arguments as to why it would be a good idea, not just try this or try that, as I'm trying to understand the whole vss setup a little better.
UPDATE : BITS writer came back in stable state after reboot, so only system writer is still missing.