Вы здесь

Creation of own service of updating: how to load files from the site

Loading
 
 
We continue our talk about creation of own service of updating in CA-Visual Objects. We already talked about archiving of data and on verification of versions of files. Time came to talk about data loading from the Internet!
 
 
For this purpose in CA-Visual Objects (CA-VO) there is a wonderful library «Internet». For operation according to the ftp protocol – we will use the class «cFTP», and for http – «cHTTP». Let's say I already have a site. I don't want to use loading according to the FTP-protocol. Means, I need to use the class «cHTTP». It seems to me that the description and examples of operation with the Internet in CA-VO it isn't enough. As speak in Russia: «без пол литра – не разобраться». I.e., to understand how it works - it is necessary to drink something alcoholic.
 
Geoff Schaller
I don't drink alcohol. Therefore, I looked for information necessary to me at forums. Very quickly I understood that in loading of files there are certain difficulties. Otherwise, why programmers try to improve and correct «Internet»-library all the time. At first the Australian Geoff Schaller programmer was engaged in it. And then is proceeded in «IC2» (see CA-Visual Objects: Third-Party Products, point 21).
 
On a photo: Geoff Schaller
 
 
The most unpleasant for beginners is an absence of good examples and the detailed description. If you don't have it - you is guaranteed you will spend a lot of time. With such situation we will be helped by SDK CA-VO.
 
I want to warn my "followers" directly: in my opinion, the code in SDK can sometimes not correspond to a real code in CA-VO. It strongly complicates understanding of the task. I.e., some errors which I found there, disappeared when instead of SDK library I used standard (not SDK) «Internet»-library. For example, in a «Help» syntax is specified:
 
CHttp{ <cCaption>, <n> }  => SELF
Init( <cCaption>, <n> )  => SELF
 
Actually, in CHttp:Init() there are three parameters (see SDK, «Internet SDK»):
 
METHOD Init( cCaption, n, lStat ) CLASS CHttp
 
Unfortunately, the third parameter isn't described. Give we will look at it. «lStat» actually is responsible for use of critical sections. TRUE is to use critical sections, FALSE – No. The author of SDK uses own mechanism of operation with sections and considers that you also use it (by default, lStat = TRUE). Therefore, by operation with SDK if you don't plan to use critical sections, you should write: cHTTP{ cCaption, n, FALSE }. Or you should rewrite SDK. The most ridiculous, in the worker (not SDK) «Internet»-library this problem isn't present. This problem exists only in SDK!
 
If you don't want to work in SDK with critical sections, I suggest making a subclass:
 
CLASS msHTTP INHERIT cHTTP // Pencil
 
METHOD Init( cCaption, n, lStat ) CLASS msHTTP // Pencil
      // Description: This method is intended only for operation with SDK!
      // Parameters :
      // Returns    :
 
      // If the critical section isn't initialized, it can't be used!
      IF csWSA.DebugInfo = NULL_PTR
            lStat := FALSE
      ENDIF
     
      SUPER:Init( cCaption, n, lStat )
 
      RETURN SELF
 
Here "csWSA" is a global variable of my critical section. And you can have another. I remind, in a normal code (not for operation with SDK) is it isn't necessary, moreover – it is contraindicated. It only for operation with SDK («Internet SDK»)!!!
 
Let's return back a little. Why I refused idea of use of ftp? First, I wanted to consider more difficult option. It can be useful if your site doesn't support loading through the FTP-protocol.
Secondly, use of HTTP allows use safer methods of communication. Thirdly, there are standard means for scanning of the files which are on ftp, and for http - No. I want that users received the required files only after processing of the form on the site. Plus, we can make very simple check of settings in Windows: if we can come into the Internet through IE – we can make loading (updating).
 
Yes… Somehow I precipitately refused idea of use of ftp… My further research showed that in standard library «Internet» there are no suitable means for operation with files according to the http-protocol. In the childhood I liked to watch newsreels. In one of them there was such phrase (I translate into English): «The nutlet of knowledge is solid. But, to us will help to split it newsreel "I want to know everything!"» And good music! What music do you listen to? And I (it is direct now) – «RPWL».
 
Plan such:
  1. We open Internet session (we transfer data about our Proxy and so forth). Here we use API function InternetOpen().
  2. We try to be connected to a Host (to ftp, http). It is InternetConnect().
  3. And we execute the necessary requests (we work in session with a Host: documents, files and so forth). See InternetOpenUrl().
 
Well, and when closing we will do everything on the contrary.
 
Let's see how and what is done in the library «Internet SDK». The first that we will see is a lack of control of actions. I.e., to us won't prevent to add check of opening of Internet session and connection with a Host (site). Plus, the analysis of the SDK code shows weak processing of closing of processes. It means that it will be better for us if we remake these slices of a code (because, we have no warranties that in standard library these issues are resolved correctly).
 
Now we have to take another important decision: how we will control and report the causes of malfunctions. I decided to use standard reception – if Windows considers that Internet session isn't open – I will simply report about it, without specifying the reason. Clearing up of the reasons is a task of the administrator. We don't write the diagnostic program? The reasons can be much: Windows (Firewall), devices (a board, the modem, a router, a cable), provider…
 
CLASS msHTTP INHERIT cHTTP // Pencil
      // Description:
 
      DECLARE ACCESS IsOpen // Status of the Internet: it is open or not
 
ACCESS IsOpen AS LOGIC PASCAL CLASS msHTTP
      // Description: Check of opening of the Internet (in case of the active or passive connection)
            // Note: if passive connection is used - this check isn't always sufficient...
 
      // Parameters : No
      // Returns    : LOGIC
 
      LOCAL lSuccess AS LOGIC
      LOCAL dwConnectionType AS DWORD
 
 
      IF SELF:hSession == NULL_PTR
            lSuccess := FALSE
      ELSE
            dwConnectionType := INTERNET_CONNECTION_MODEM + INTERNET_CONNECTION_LAN + INTERNET_CONNECTION_PROXY 
 
                  // INTERNET_CONNECTION_MODEM      = 1; // Connection on the modem
                  // INTERNET_CONNECTION_LAN        = 2; 
                  // INTERNET_CONNECTION_PROXY      = 4; 
/*
; Return :
; 0x40 INTERNET_CONNECTION_CONFIGURED : Local system has a valid connection To the Internet, but it might Or might not be currently connected.
; 0x02 INTERNET_CONNECTION_LAN : Local system uses a Local area network To connect To the Internet.
; 0x01 INTERNET_CONNECTION_MODEM : Local system uses a modem To connect To the Internet.
; 0x08 INTERNET_CONNECTION_MODEM_BUSY : No longer used.
; 0x20 INTERNET_CONNECTION_OFFLINE : Local system is in offline mode.
; 0x04 INTERNET_CONNECTION_PROXY : Local system uses a proxy server To connect To the Internet.
; 0x10 INTERNET_RAS_INSTALLED : Local system has RAS installed
; or 0 if there is no Internet connection
*/
            lSuccess := InternetGetConnectedState( @dwConnectionType, 0 )
      ENDIF
 
      RETURN lSuccess
 
METHOD IsConnected( cHost ) CLASS msHTTP
      // Description: Check of connection with a Host
      // Parameters :
      // Returns    : LOGIC
 
      LOCAL lSuccess AS LOGIC
 
 
      IF SELF:hConnect == NULL_PTR
            lSuccess := FALSE
      ELSE
 
            IF     IsNil( cHost )
 
                  cHost := SELF:cCurrentUrl
                  lSuccess := InternetCheckConnection( String2Psz( cHost ), FLAG_ICC_FORCE_CONNECTION, 0 )
 
            ELSEIF IsString( cHost )
 
                  cHost := AllTrim( cHost )
                  IF cHost == NULL_STRING
                        cHost := SELF:cCurrentUrl
                  ELSE
 
                  IF At2( "//", cHost ) == 0
 
                     IF SELF:lFtpRequest
                        cHost := "ftp://" + cHost
                     ELSE
                        cHost := "http://" + cHost
                     ENDIF
                  ENDIF
                  ENDIF
 
                  lSuccess := InternetCheckConnection( String2Psz( cHost ), FLAG_ICC_FORCE_CONNECTION, 0 )
            ELSE
                  lSuccess := FALSE
            ENDIF
      ENDIF
 
      RETURN lSuccess
 
METHOD CloseRequest( ) CLASS msHTTP
      // Description: Closing of request (requests)
      // Parameters : No
      // Returns    : NIL
 
      LOCAL lRet AS LOGIC
 
 
lRet := TRUE
 
      IF SELF:hRequest == NULL_PTR
            RETURN lRet
      ENDIF
 
// We delete request in critical section
      IF SELF:lStatus
            lRet := SELF:__DelStatus( SELF:hRequest )
      ENDIF
 
      IF lRet .AND. InternetCloseHandle( SELF:hRequest )
 
            SELF:hRequest := NULL_PTR
            RETURN TRUE
      ENDIF
 
      RETURN FALSE
 
METHOD CloseRemote( ) CLASS msHTTP
      // Description: Closing of connection with the site
      // Parameters :
      // Returns    :
 
      LOCAL lRet AS LOGIC
 
 
// We close request
      lRet := SELF:CloseRequest( )
 
      IF SELF:hConnect == NULL_PTR
            RETURN lRet
      ENDIF
 
// We clean connection critical section
      IF SELF:lStatus
            lRet := SELF:__DelStatus( SELF:hConnect )
      ENDIF
 
      IF lRet .AND. InternetCloseHandle( SELF:hConnect )
 
            SELF:hConnect := NULL_PTR
            RETURN TRUE
      ENDIF
 
      RETURN FALSE
 
METHOD Close( ) CLASS msHTTP
      // Description: Closing of an Internet-session
      // Parameters : No
      // Returns    : NIL
 
 
      // We close connection
      IF.NOT.SELF:CloseRemote()
 
// !!! If it wasn't closed - we close as we can
 
            // We close request
            IF .NOT. SELF:hRequest == NULL_PTR
 
                  IF SELF:lStatus
                        SELF:__DelStatus( SELF:hRequest )
                  ENDIF
                  InternetCloseHandle( SELF:hRequest )
                  SELF:hRequest := NULL_PTR
            ENDIF
 
            // We close connection
            IF .NOT. SELF:hConnect == NULL_PTR
 
                  IF SELF:lStatus
                        SELF:__DelStatus( SELF:hConnect )
                  ENDIF
                  InternetCloseHandle( SELF:hConnect )
                  SELF:hConnect := NULL_PTR
            ENDIF
      ENDIF
 
// We close the Internet
      IF .NOT. SELF:hSession == NULL_PTR
 
            IF SELF:lStatus
                  SELF:__DelStatus( SELF:hSession )
            ENDIF
            InternetCloseHandle( SELF:hSession )
            SELF:hSession := NULL_PTR
      ENDIF
 
// We clean variables
      SELF:cResponse               := NULL_STRING
      SELF:cResponseHeader    := NULL_STRING
      SELF:cCurDir                 := NULL_STRING
      SELF:cCurrentUrl        := NULL_STRING
 
      SELF:cHostAddress       := NULL_STRING
      SELF:cUserName               := NULL_STRING
      SELF:cPassWord               := NULL_STRING
      SELF:cError                  := NULL_STRING
      SELF:cAgent                  := NULL_STRING
      SELF:cProxy                  := NULL_STRING
      SELF:cProxyBypass       := NULL_STRING
 
      RETURN NIL
 
METHOD Axit( ) CLASS msHTTP
      // Description:
      // Parameters : No
      // Returns    : NIL
  
      SELF:Close()
 
      RETURN ( SUPER:Axit() )
 
 
So, how to open – we know how to close – already wrote. It remains to learn how to download the files.
-         How many it is already possible to scoff at readers?
-         It is necessary to work a little more and to rewrite the standard GetFile() method. Now we will pick up to it «suit» and «shoes»:
 
METHOD GetFile( cRemoteFile, cNewFile, lFailIfExists ) CLASS msHTTP // Pencil
      // Description: Loading the specified file
      // Parameters :
      // Returns    : LOGIC
 
      LOCAL lRet AS LOGIC
 
      LOCAL cHead AS STRING
      LOCAL nFlags AS DWORD
      LOCAL cUrl AS STRING
 
      LOCAL hNewFile AS PTR
 
      LOCAL DIM abTemp[ MAX_SOCKBUFF ] AS BYTE
      LOCAL nSize AS DWORD
 
      LOCAL lFile AS LOGIC
 
 
      Default( @lFailIfExists,     FALSE )
      Default( @cRemoteFile,       NULL_STRING )
      Default( @cNewFile,                NULL_STRING )
 
      cRemoteFile := AllTrim( cRemoteFile )
      cNewFile    := AllTrim( cNewFile )
 
lRet := FALSE
 
      IF cRemoteFile == NULL_STRING
            RETURN lRet
      ENDIF
 
      IF cNewFile == NULL_STRING
            cNewFile := cRemoteFile
      ENDIF
 
      lFile := File( cNewFile )
 
// If the file exists, but we can't write
      IF lFile .AND. lFailIfExists
            RETURN lRet
      ENDIF
 
// We begin preparation
      IF SELF:hSession == NULL_PTR
 
            IF .NOT. SELF:Open( NIL, SELF:cProxy, SELF:cProxyBypass )
                  RETURN lRet
            ENDIF
      ENDIF
 
      cUrl := SELF:__GetUrl( cRemoteFile )
      IF cUrl == NULL_STRING
            RETURN lRet
      ENDIF
 
      cHead       := HEADER_ACCEPT
      nFlags      := INTERNET_FLAG_DONT_CACHE + INTERNET_FLAG_EXISTING_CONNECT
 
// !!!
SELF:__SetStatusObject()
 
      SELF:hRequest := InternetOpenUrl( SELF:hSession, String2Psz( cUrl ), String2Psz( cHead ), SLen( cHead ), nFlags, SELF:__GetStatusContext() )
      IF .NOT. SELF:hRequest == NULL_PTR
 
SELF:__SetStatus( SELF:hRequest )
 
            // We create the file
            IF lFile
                  hNewFile := FxOpen( cNewFile, OF_READWRITE + OF_SHARE_EXCLUSIVE, NULL_STRING )
            ELSE
                  hNewFile := FCreate2( cNewFile, FC_ARCHIVED )
            ENDIF
 
            IF .NOT. hNewFile == INVALID_HANDLE_VALUE // F_ERROR
 
                  // We read and write to the file
                  DO WHILE ( lRet := InternetReadFile( SELF:hRequest, @abTemp[ 1 ], MAX_SOCKBUFF, @nSize ) )
 
                        IF nSize = 0
                             EXIT
                        ENDIF
 
                        FWrite3( hNewFile, @abTemp[ 1 ], nSize )
                  END DO
 
                  // We close the file
                  FClose( hNewFile )
                  hNewFile := NULL_PTR
 
                  // If bad - we delete the file
                  IF !lRet
                        FErase( cNewFile )
                  ENDIF
            ENDIF
 
// We release Internet-request
SELF:__DelStatus(SELF:hRequest)
 
            IF InternetCloseHandle( SELF:hRequest )
                  SELF:hRequest := NULL_PTR
            ENDIF
      ENDIF
 
SELF:__DelStatusObject()
 
      RETURN lRet
 
 
I want to warn impatient testers: be attentive! My code isn't ideal and doesn't consider many nuances. For example, the method of obtaining files doesn't consider existence of such opportunity, as URL redirection. I.e., if the user receives a certain link which actually will redirect him to other place where he will receive following (perhaps, correct) – my GetFile() method won't work.
 
Speak, such problem can be solved by viewing and analyzing the response header (see the GetResponseHeader() method). Once again, be attentive, and all in your hands!
 
 
Example "on snack":
 
      oHTTP := msHTTP{ }
 
      IF oHTTP:Open( ) // Opened Internet-session
 
            IF oHTTP:ConnectRemote( "Test.Com" ) // Name or IP address of the site
                  cRemFile := "sites/default/files/Test.rar" // Here we specify a full path from site "root"
                  cNewFile := "D:\Demo.rar"
                  IF oHTTP:GetFile( cRemFile, cNewFile )
 
                        ErrorBox{, "We will succeed !"}:Show()
                  ELSE
                        ErrorBox{, "Error !"}:Show()
                  ENDIF
            ELSE
                  ErrorBox{, " Error !"}:Show()
            ENDIF
      ELSE
            ErrorBox{, " Error !"}:Show()
      ENDIF
 
      oHTTP:Close()
 
 
      Good luck!
 
      09.11.2014 г.      Pencil (Карандаш).
 

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer