WebSphere nodes no longer syncing
Came across this issue recently, took em a while to get it solved. None of the nodes could sync with the deployment manager due to following error:
ADFS0119E: An unexpected exception occurred: java.io.FileNotFoundException: File download failed: java.io.IOException: ADFS0140E: The file transfer server cannot download the download%2Fcells%2FSocialCell%2FLotusConnections-config%2F%0D__5655077188255192541.tmp file. The server returned HTTP error code 500.
at com.ibm.websphere.management.filetransfer.client.FileDownloadInputStream.getStream(FileDownloadInputStream.java:230)
at com.ibm.websphere.management.repository.RepositoryInputStream.read(RepositoryInputStream.java:137)
at java.security.DigestInputStream.read(DigestInputStream.java:112)
at java.io.FilterInputStream.read(FilterInputStream.java:101)
at com.ibm.ws.management.repository.FileDocument.copyDocument(FileDocument.java:452)
at com.ibm.ws.management.repository.FileDocument.create(FileDocument.java:118)
at com.ibm.ws.management.repository.FileRepository.createInternal(FileRepository.java:775)
at com.ibm.ws.management.repository.FileRepository.updateInternal(FileRepository.java:1481)
at com.ibm.ws.management.repository.FileRepository.update(FileRepository.java:1397)
at com.ibm.ws.management.repository.FileRepository.create(FileRepository.java:709)
at com.ibm.ws.management.repository.FileRepository.create(FileRepository.java:698)
at com.ibm.ws.management.sync.NodeSyncTask.processFolderSyncUpdate(NodeSyncTask.java:457)
at com.ibm.ws.management.sync.NodeSyncTask.doSync(NodeSyncTask.java:283)
This got me puzzled for a while, I searched for a file starting with D_ in the Deployment managers LotusConnections-config directory but could not find anything.
I deleted profiles/dmgr/wstemp/, profiles/dmgr/temp/, profiles/dmgr/config/temp/, no change.
I then manually ran the syncnode command using the -trace option to generate some more logging.
This is what I got:
[2/25/15 10:55:28:451 CET] 00000002 FileTransferC < openConnection Exit com.ibm.net.ssl.www2.protocol.https.e:https://hostname: 9043/FileTransfer/transfer/download%2Fcells%2FSocialCell%2FLotusConnecti ons-config%2F%0D__8828909330741519452.tmp?compress=true&deleteOnCompleti on=true [2/25/15 10:55:28:477 CET] 00000002 FileTransferU > streamToString
Entry
[2/25/15 10:55:28:479 CET] 00000002 FileTransferU < streamToString Exit
Error 500:
java.lang.IllegalArgumentException: Invalid CR not followed by LF
Not really getting me more info, except that I was still having an issue with a file starting with D_
But after having another look at the LotusConnections-config directory I saw something I did not realize before.
In the file list I had a file called ? with a size of 0 bytes
This seems to be an issue for the sync process, it cannot handle that file. Not sure how it got there, but after removing it sync ran fine