Tuesday, May 7, 2013

SSIS: An error occurred in the requested FTP operation. Detailed error description: The password was not allowed

SSIS Error:
An error occurred in the requested FTP operation. Detailed error description: The password was not allowed:

Requirement: was to develop a SSIS package for reading MOST RECENT file from FTP server and load data to a Table in Database.

Solution consist of following steps:
1. Starting with script task which will read the file name listing from FTP server and identify latest file in the list of file names. (Script can be found at the end )
2. Check if file is already processed.
3. If it is new file use FTP task to move the file from FTP to local staging folder.
4. Move the existing data from destination table to a history table with the file name with the data.
5.Clear all the data from destination (an alternate approach have been to just to upsert to so that we don't have to truncate the whole table and just go with the update, in this scenario requirement was to reflect data from latest file only)
6. Load the data from this new file into destination table.

One of the challenge was to read FTP configuration only once and then use these values in script task as well as regular FTP task, It would have been easier to read it twice from configuration, but was not resulting in elegant solution and more over it could have been maintenance nightmare.

Approach was taken to create parameter  like FTP server name and FTP user name and  password and read  values from configuration only once.

upon further search found way to use these parameter to create connection string for FTP connetor for FTP task. (http://social.msdn.microsoft.com/Forums/en-US/sqlintegrationservices/thread/3713e9a5-343a-4c5b-ac5d-1508cd5ab8be)

Connection string = @[User::FTPServer]+":21" + "." + @[User::FTPUser] + "." + @[User::FTPPass]

In my scenario since I was reading FTPServer value as = ftp://ftp1.servname.com

I had to use following expression
Connection string = REPLACE(@[User::FTPServer] ,"ftp://","")+":21"+"."+@[User::FTPUserName]+"."+ @[User::FTPPwd]

these program ran without any error in BIDS in debug mode, but when I deployed to prod server I started to get this error:


Error: 2013-05-07 16:06:15.06
   Code: 0xC0016016
   Source:
   Description: Failed to decrypt protected XML node "DTS:Property" with error 0x8009000B "Key not valid for use in specified state.". You may not be authorized to access this information. This error occurs when there is a cryptographic error. Verify that the correct key is available.
End Error
Error: 2013-05-07 16:06:21.56
   Code: 0xC001602A
   Source: pakage_name Connection manager "FTP Connection Manager"
   Description: An error occurred in the requested FTP operation. Detailed error description: The password was not allowed.
End Error
Error: 2013-05-07 16:06:21.59
   Code: 0xC002918F
   Source: FTP Task Move the file to local folder FTP Task
   Description: Unable to connect to FTP server using "FTP Connection Manager".
End Error
DTExec: The package execution returned DTSER_FAILURE (1).


SOLUTION TIME:
Upon further search I found the solution (http://social.technet.microsoft.com/Forums/en-US/sqlintegrationservices/thread/912d43a5-975f-48b6-a9f1-660efeacc8ed/)


Job gets working by changing the ProtectionLevel on the FTP packages to EncryptSensitiveWithPassword.


:OM:

Following is code is used to read latest file name from FTP server:


        public void Main()        {                        string strFileNameLike= Dts.Variables["FileNameLike"].Value.ToString();            string strFTPServer= Dts.Variables["FTPServer"].Value.ToString();            string strFTPUserName= Dts.Variables["FTPUserName"].Value.ToString();            string strFTPPwd= Dts.Variables["FTPPwd"].Value.ToString();            string strFTPFolder= Dts.Variables["FTPFolder"].Value.ToString();            string strFTPFullPath=strFTPServer+"/"+strFTPFolder;               FtpWebRequest request = (FtpWebRequest)FtpWebRequest.Create(strFTPFullPath);            request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;            request.Credentials = new NetworkCredential(strFTPUserName, strFTPPwd);            request.UsePassive = true;            request.UseBinary = true;            request.KeepAlive = false;             string[] list = null;            string[] listFilteredArray = null;             List<String> listFiltered = new List<string>();              using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())            using (StreamReader reader = new StreamReader(response.GetResponseStream()))            {                list = reader.ReadToEnd().Split(new string[] { "\r\n" }, StringSplitOptions.RemoveEmptyEntries);            }             string strStartsWith = strFileNameLike.Split(new string[] {"*"}, StringSplitOptions.RemoveEmptyEntries)[0].ToString();            string strEndsWith = strFileNameLike.Split(new string[] { "*" }, StringSplitOptions.RemoveEmptyEntries)[1].ToString();              foreach (string strname in list)            {
                if(strname.Substring(56, 25).StartsWith(strStartsWith) && strname.Substring(56, 25).EndsWith(strEndsWith) )                {                    listFiltered.Add(strname.Substring(56, 25));
                }         
            } 
            listFilteredArray= listFiltered.ToArray(); 
                        if (listFilteredArray.Length > 0)            {                Dts.Variables["FileName"].Value = listFilteredArray[listFilteredArray.Length - 1].ToString();            }
            Dts.TaskResult = (int)ScriptResults.Success;        }

No comments: