Problem

After upgrading your TSX SQL Server 2005/2008 to SQL Server 2008 R2, you may experience some connectivity issues between your newly upgraded TSX server and your MSX server.  If you see the below error in the SQL Server Agent log, then this article may be of use to you: –

Unable to connect to MSX ‘ServerName\InstanceName’.

SQLServer Error: 802, Client unable to establish connection.

SQLServer Error: 802, SSL Provider: The target principal name is incorrect.

Cause

This can occur because you used Servername\InstanceName instead of the actual fully qualified name specified in the certificate.

Resolution

To resolve this issue, update the column ‘originating_server’ in table msdb.dbo.sysoriginatingservers to use the fully qualified name of the server i.e. myservername.domainname.co.uk or if it’s an instance myservername.domainname.co.uk\instancename.

You will also need to update the registry key value  HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL10_50.InstanceName\SQLServerAgent\MSXServerName

Restart the SQL Server Agent and review the log. You may still have some issues downloading the instructions, but that is easily resolved.