logging_xml » Revisions »

Differences

This shows you the differences between the selected revisions of the page.

logging_xml 2013-11-05 logging_xml 2022-06-16 (current)
Line 1: Line 1:
====== XML Logging ====== ====== XML Logging ======
-XML logging is one of the WinSCP [[logging|log formats]]. XML log includes structured records describing operations done by WinSCP over session. The log format is protocol independent.+XML logging is one of the WinSCP [[logging|log formats]]. An %%XML%% log includes structured records describing operations done by WinSCP over session. The log format is protocol independent.
-In many cases you do not have to deal with the XML log file directly. [[library|WinSCP .NET assembly]] can do it for you.+In many cases you do not have to deal with the %%XML%% log file directly. [[library|WinSCP .NET assembly]] can do it for you.
-===== Purpose =====+===== [[purpose]] Purpose =====
Primary purpose of the XML logging is to provide machine-readable description of automatically performed operations, such as when using [[scripting|scripting]]. Primary purpose of the XML logging is to provide machine-readable description of automatically performed operations, such as when using [[scripting|scripting]].
-XML logging is useful e.g. to: +The %%XML%% logging is useful e.g. to: 
-  * Find a list of files that were actually uploaded/downloaded, when transferring whole directory or files matching a mask; +  * Find a list of files that were actually uploaded/downloaded, when transferring a whole directory or when transferring files matching a mask; 
-  * Get directory listing;+  * Get a directory listing;
  * Record operations done during [[task_synchronize_full|synchronization]].   * Record operations done during [[task_synchronize_full|synchronization]].
-===== Using ===== +Note that while the %%XML%% logging can be used also for GUI sessions, it does not record [[transfer_queue|background transfers]].
-XML logging, being used along with [[scripting|scripting]], is most typically enabled from [[commandline|command-line]], using ''/xmllog'' parameter. It can be enabled in [[ui_pref_logging|preferences]] too.+
-Overall format of the XML log file follows:+===== [[using]] Using ===== 
 +The %%XML%% logging, being used along with [[scripting|scripting]], is most typically enabled from command-line, using ''[[commandline#logging|/xmllog]]'' parameter. It can be enabled in [[ui_pref_logging|preferences]] too. 
 + 
 +An overall format of the %%XML%% log file follows:
<code xml> <code xml>
Line 25: Line 27:
</code> </code>
-The top level ''session'' tag represents one logical session, which may consist of several physical sessions, particularly when connection is lost. Attribute ''name'' refers to name of the logical session. Attribute ''start'' indicates time when the session was initiated·((All timestamps in XML log have XML ''dateTime'' type, where only ''%%YYYY-MM-DD"T"HH:MM:SS.NNN"Z"%%'' syntax is used.)).+The top level ''session'' tag represents one logical session, which may consist of several physical sessions, particularly when connection is lost. Attribute ''name'' refers to name of the logical session. Attribute ''start'' indicates time when the session was initiated.((All timestamps in the %%XML%% log have %%XML%% ''dateTime'' type, where only ''%%YYYY-MM-DD"T"HH:MM:SS.NNN"Z"%%'' syntax is used.))
-The ''session'' element includes [[logging_xml#elements|child elements]], where each element represents single log entry, i.e. single physical operation with remote file over the logical session.+The ''session'' element includes [[#elements|child elements]], where each element represents single log entry, i.e. single physical operation with remote file over the logical session.
===== [[operations]] Representing Operations in Log ===== ===== [[operations]] Representing Operations in Log =====
-Every entry in XML log represents single physical operation over the session. Typically this would be an operation with remote file.+Every entry in the %%XML%% log represents single physical operation over the session. Typically this would be an operation with remote file.
Single logical operation (in [[scripting|scripting]] or GUI) may actually involve multiple physical operations, each represented with separate log element. For example [[task_upload|upload]] of file (e.g. using ''put'' scripting command) may be represented by up to three elements, ''upload'' for actual upload, ''touch'' for preserving timestamp and ''chmod'' for preserving permissions. Single logical operation (in [[scripting|scripting]] or GUI) may actually involve multiple physical operations, each represented with separate log element. For example [[task_upload|upload]] of file (e.g. using ''put'' scripting command) may be represented by up to three elements, ''upload'' for actual upload, ''touch'' for preserving timestamp and ''chmod'' for preserving permissions.
-Note that some logical operations does not have corresponding log element (e.g. [[task_link|creation of symbolic link]]). Such operations are omitted from the XML log.+Note that some logical operations does not have corresponding log element (e.g. [[task_link|creation of symbolic link]]). Such operations are omitted from the %%XML%% log.
===== [[result]] Representing Results/Errors ===== ===== [[result]] Representing Results/Errors =====
Line 58: Line 60:
</code> </code>
-With some protocols, each of the physical operations are performed individually. With some protocols, set of operations may be performed in atomic form. This may prevent mapping error to specific operation. In this case the error may be associated with more operations, resulting in its duplication in the XML log.+With some protocols, each of the physical operations are performed individually. With some protocols, set of operations may be performed in atomic form. This may prevent mapping error to specific operation. In this case the error may be associated with more operations, resulting in its duplication in the %%XML%% log.
Some errors may not be associated with any operation in the XML log. This particularly happens when: Some errors may not be associated with any operation in the XML log. This particularly happens when:
Line 65: Line 67:
==== [[result_script]] Results/Errors of Script ==== ==== [[result_script]] Results/Errors of Script ====
-When the script fails (WinSCP [[scripting#using_scripting|exit code]] is 1), the top level ''session'' element or ''group'' element (when [[logging_xml#group|grouping]] is enabled) will include one or more ''failure'' elements, each containing one or more ''message'' elements with error message(s).+When the script fails (WinSCP [[scripting#using_scripting|exit code]] is 1), the top level ''session'' element or ''group'' element (when [[#group|grouping]] is enabled) will include one or more ''failure'' elements, each containing one or more ''message'' elements with error message(s).
If the particular error was associated with a physical operation, the error message will be included both in ''result'' child element of the respective operation element and in ''failure'' child element of top level ''session'' element or ''group'' element. If the particular error was associated with a physical operation, the error message will be included both in ''result'' child element of the respective operation element and in ''failure'' child element of top level ''session'' element or ''group'' element.
Line 90: Line 92:
===== [[group]] Grouping Operations for Commands ===== ===== [[group]] Grouping Operations for Commands =====
-When using XML logging together with [[scripting|scripting]], [[logging_xml#elements|operations]] and [[logging_xml#result_script|failures]] belonging to the same command can be groupped using parent ''group'' element:+When using the %%XML%% logging together with [[scripting|scripting]], [[#elements|operations]] and [[#result_script|failures]] belonging to the same command can be grouped using parent ''group'' element:
<code xml> <code xml>
-<group name="put -preservetime d:\*.txt" start="2012-01-04T14:25:10.204Z">+<group name="put -preservetime d:\*.txt" start="2021-12-03T14:25:10.204Z">
  <upload>   <upload>
    <filename value="d:\readme.txt" />     <filename value="d:\readme.txt" />
    <destination value="/home/user/readme.txt" />     <destination value="/home/user/readme.txt" />
 +    <size value="15345" />
    <result success="true" />     <result success="true" />
  </upload>   </upload>
Line 124: Line 127:
===== [[elements]] Elements ===== ===== [[elements]] Elements =====
-All operation elements below have ''[[logging_xml#result|result]]'' child element in addition to listed child elements.+All operation elements below have ''[[#result|result]]'' child element in addition to listed child elements.
The ''result'' child element is always present. Many other child elements may be absent in case of error. The ''result'' child element is always present. Many other child elements may be absent in case of error.
-==== call ====+==== [[call]] call ====
Execution of arbitrary [[remote_command|remote shell command]] (with [[protocols|SFTP and SCP protocols]]) or execution of a protocol command (with FTP protocol). Execution of arbitrary [[remote_command|remote shell command]] (with [[protocols|SFTP and SCP protocols]]) or execution of a protocol command (with FTP protocol).
Line 154: Line 157:
Associated script commands: ''[[scriptcommand_call|call]]'' Associated script commands: ''[[scriptcommand_call|call]]''
-==== chmod ====+==== [[checksum]] checksum ==== 
 +[[ui_properties#checksum|Calculating a checksum]] of a remote file.  
 + 
 +Elements: 
 +^ Element                  ^ Description ^ 
 +| ''filename''            | Absolute path to a remote file (in ''value'' attribute) | 
 +| ''algorithm''            | Name of a checksum algorithm used (in ''value'' attribute) | 
 +| ''checksum''            | Hex dump of a calculated checksum (in ''value'' attribute) | 
 + 
 +Example: 
 +<code xml> 
 +<checksum> 
 +  <filename value="/home/martin/public_html/index.html" /> 
 +  <algorithm value="sha-1" /> 
 +  <checksum value="bb4dfa9b51d3f6c99b5ec6c12ebf9cade79f43c4" /> 
 +  <result success="true" /> 
 +</checksum> 
 +</code> 
 + 
 +Associated script commands: ''[[scriptcommand_checksum|checksum]]'' 
 + 
 +==== [[chmod]] chmod ====
[[task_properties|Changing of permissions]] of one (or more) remote file. [[task_properties|Changing of permissions]] of one (or more) remote file.
Line 160: Line 184:
^ Element                  ^ Description ^ ^ Element                  ^ Description ^
| ''filename''            | Absolute path to remote file (in ''value'' attribute) | | ''filename''            | Absolute path to remote file (in ''value'' attribute) |
-| ''permissions''          | Permissions in Unix format ''rwxrwxrwx'' |+| ''permissions''          | Permissions in Unix format ''rwxrwxrwx'' (in ''value'' attribute) |
With [[scp|SCP protocol]] optional boolean attribute ''recursive'' indicates, if permissions were changed recursively with single operation. With [[scp|SCP protocol]] optional boolean attribute ''recursive'' indicates, if permissions were changed recursively with single operation.
Line 175: Line 199:
Associated script commands: ''[[scriptcommand_chmod|chmod]]'', ''[[scriptcommand_keepuptodate|keepuptodate]] -permissions'', ''[[scriptcommand_put|put]] -permissions'', ''[[scriptcommand_synchronize|synchronize]] remote|both -permissions'' Associated script commands: ''[[scriptcommand_chmod|chmod]]'', ''[[scriptcommand_keepuptodate|keepuptodate]] -permissions'', ''[[scriptcommand_put|put]] -permissions'', ''[[scriptcommand_synchronize|synchronize]] remote|both -permissions''
-==== download ====+==== [[cp]] cp ==== 
 +[[task_move_duplicate#duplicate|Duplication]] of one remote file or directory. 
 + 
 +Elements: 
 +^ Element                  ^ Description ^ 
 +| ''filename''            | Absolute path to source local file (in ''value'' attribute) | 
 +| ''destination''          | Absolute path to destination remote file (in ''value'' attribute) | 
 + 
 +Example: 
 +<code xml> 
 +<cp> 
 +  <filename value="/home/martin/public_html/about.html" /> 
 +  <destination value="/home/martin/backup/about.html.20171220" /> 
 +  <result success="true" /> 
 +</cp> 
 +</code> 
 + 
 +Associated script commands: ''[[scriptcommand_cp|cp]]'' 
 + 
 +==== [[download]] download ====
[[task_download|Downloading]] of single file. [[task_download|Downloading]] of single file.
Line 182: Line 225:
| ''filename''            | Absolute path to source remote file (in ''value'' attribute) | | ''filename''            | Absolute path to source remote file (in ''value'' attribute) |
| ''destination''          | Absolute path to destination local file (in ''value'' attribute)((File name may differ from name of source file.)) | | ''destination''          | Absolute path to destination local file (in ''value'' attribute)((File name may differ from name of source file.)) |
 +| ''size''                | Number of bytes transferred. |
Example: Example:
Line 188: Line 232:
  <filename value="/home/martin/public_html/about.html" />   <filename value="/home/martin/public_html/about.html" />
  <destination value="d:\www\about.htm" />   <destination value="d:\www\about.htm" />
 +  <size value="55387" />
  <result success="true" />   <result success="true" />
</download> </download>
Line 194: Line 239:
Associated script commands: ''[[scriptcommand_get|get]]'', ''[[scriptcommand_synchronize|synchronize]] local|both'' Associated script commands: ''[[scriptcommand_get|get]]'', ''[[scriptcommand_synchronize|synchronize]] local|both''
-==== ls ====+==== [[ls]] ls ====
Listing of remote directory (only when explicitly requested using ''[[scriptcommand_ls|ls]]'' scripting command). Listing of remote directory (only when explicitly requested using ''[[scriptcommand_ls|ls]]'' scripting command).
Line 207: Line 252:
| ''type''                | Type of file as in Unix ''ls'' command output, e.g. ''d'' for directory (in ''value'' attribute) | | ''type''                | Type of file as in Unix ''ls'' command output, e.g. ''d'' for directory (in ''value'' attribute) |
| ''size''                | Size of file in bytes (in ''value'' attribute) | | ''size''                | Size of file in bytes (in ''value'' attribute) |
-| ''modification''        | Modification timestamp (in ''value'' attribute)((All timestamps in XML log have XML ''dateTime'' type, where only ''%%YYYY-MM-DD"T"HH:MM:SS.NNN"Z"%%'' syntax is used.)) |+| ''modification''        | Modification timestamp (in ''value'' attribute)((All timestamps in the %%XML%% log have %%XML%% ''dateTime'' type, where only ''%%YYYY-MM-DD"T"HH:MM:SS.NNN"Z"%%'' syntax is used.)) |
| ''permissions''          | File permissions in Unix format ''rwxrwxrwx'' (in ''value'' attribute) | | ''permissions''          | File permissions in Unix format ''rwxrwxrwx'' (in ''value'' attribute) |
 +| ''owner''                | File owner (in ''value'' attribute) |
 +| ''group''                | File group (in ''value'' attribute) |
Example: Example:
Line 248: Line 295:
Associated script commands: ''[[scriptcommand_ls|ls]]'' Associated script commands: ''[[scriptcommand_ls|ls]]''
-==== mkdir ====+==== [[mkdir]] mkdir ====
[[task_create_directory|Creating of remote directory]]. [[task_create_directory|Creating of remote directory]].
Line 265: Line 312:
Associated script commands: ''[[scriptcommand_keepuptodate|keepuptodate]]'', ''[[scriptcommand_mkdir|mkdir]]'', ''[[scriptcommand_put|put]]'', ''[[scriptcommand_synchronize|synchronize]] remote|both'' Associated script commands: ''[[scriptcommand_keepuptodate|keepuptodate]]'', ''[[scriptcommand_mkdir|mkdir]]'', ''[[scriptcommand_put|put]]'', ''[[scriptcommand_synchronize|synchronize]] remote|both''
-==== mv ==== +==== [[mv]] mv ==== 
-[[task_move_duplicate#move|Moving]] or of one remote file or directory to different remote directory or [[task_rename|renaming]] of one remote file or directory.+[[task_move_duplicate#move|Moving]] of one remote file or directory to different remote directory or [[task_rename|renaming]] of one remote file or directory.
Elements: Elements:
Line 284: Line 331:
Associated script commands: ''[[scriptcommand_mv|mv]]'' Associated script commands: ''[[scriptcommand_mv|mv]]''
-==== rm ====+==== [[rm]] rm ====
[[task_delete|Deleting]] of one (or more) remote file or directory. [[task_delete|Deleting]] of one (or more) remote file or directory.
Line 303: Line 350:
Associated script commands: ''[[scriptcommand_get|get]] -delete'', ''[[scriptcommand_keepuptodate|keepuptodate]] -delete'', ''[[scriptcommand_rm|rm]]'', ''[[scriptcommand_rmdir|rmdir]]'', ''[[scriptcommand_synchronize|synchronize]] local|both -delete'' Associated script commands: ''[[scriptcommand_get|get]] -delete'', ''[[scriptcommand_keepuptodate|keepuptodate]] -delete'', ''[[scriptcommand_rm|rm]]'', ''[[scriptcommand_rmdir|rmdir]]'', ''[[scriptcommand_synchronize|synchronize]] local|both -delete''
-==== stat ====+==== [[stat]] stat ====
Listing of remote file attributes. Listing of remote file attributes.
Line 309: Line 356:
^ Element                  ^ Description ^ ^ Element                  ^ Description ^
| ''filename''            | Absolute path to remote file (in ''value'' attribute) | | ''filename''            | Absolute path to remote file (in ''value'' attribute) |
-| ''file''                | Container of file attributes. See specification in description of ''[[logging_xml#ls|ls]]'' element. Note that ''file'' element within ''stat'' misses ''filename'' subelement, comparing to ''file'' element within ''ls''/''files''. |+| ''file''                | Container of file attributes. See specification in description of ''[[#ls|ls]]'' element. Note that ''file'' element within ''stat'' misses ''filename'' subelement, comparing to ''file'' element within ''ls''/''files''. |
Example: Example:
Line 327: Line 374:
Associated script commands: ''[[scriptcommand_stat|stat]]'' Associated script commands: ''[[scriptcommand_stat|stat]]''
-==== upload ====+==== [[upload]] upload ====
[[task_upload|Uploading]] of single file. [[task_upload|Uploading]] of single file.
Line 334: Line 381:
| ''filename''            | Absolute path to source local file (in ''value'' attribute) | | ''filename''            | Absolute path to source local file (in ''value'' attribute) |
| ''destination''          | Absolute path to destination remote file (in ''value'' attribute)((File name may differ from name of source file.)) | | ''destination''          | Absolute path to destination remote file (in ''value'' attribute)((File name may differ from name of source file.)) |
 +| ''size''                | Number of bytes transferred. |
Example: Example:
Line 340: Line 388:
  <filename value="d:\www\about.htm" />   <filename value="d:\www\about.htm" />
  <destination value="/home/martin/public_html/about.html" />   <destination value="/home/martin/public_html/about.html" />
 +  <size value="55387" />
  <result success="true" />   <result success="true" />
</upload> </upload>
Line 346: Line 395:
Associated script commands: ''[[scriptcommand_keepuptodate|keepuptodate]]'', ''[[scriptcommand_put|put]]'', ''[[scriptcommand_synchronize|synchronize]] remote|both'' Associated script commands: ''[[scriptcommand_keepuptodate|keepuptodate]]'', ''[[scriptcommand_put|put]]'', ''[[scriptcommand_synchronize|synchronize]] remote|both''
-==== touch ====+==== [[touch]] touch ====
Changing of remote file timestamp. Changing of remote file timestamp.
Line 352: Line 401:
^ Element                  ^ Description ^ ^ Element                  ^ Description ^
| ''filename''            | Absolute path to remote file (in ''value'' attribute) | | ''filename''            | Absolute path to remote file (in ''value'' attribute) |
-| ''modification''        | Modification timestamp (in ''value'' attribute)((All timestamps in XML log have XML ''dateTime'' type, where only ''%%YYYY-MM-DD"T"HH:MM:SS.NNN"Z"%%'' syntax is used.)) |+| ''modification''        | Modification timestamp (in ''value'' attribute)((All timestamps in the %%XML%% log have %%XML%% ''dateTime'' type, where only ''%%YYYY-MM-DD"T"HH:MM:SS.NNN"Z"%%'' syntax is used.)) |
Example: Example:
Line 366: Line 415:
===== Schema ===== ===== Schema =====
-XML schema for the XML log is available at:+An %%XML%% schema for the %%XML%% log is available at:
http://winscp.net/schema/session/1.0 http://winscp.net/schema/session/1.0
===== Example ===== ===== Example =====
-//Note that the ''[[logging_xml#group|group]]'' elements will be included, if enabled only.//+//Note that the ''[[#group|group]]'' elements will be included, if enabled only.//
<code xml> <code xml>
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<session xmlns="http://winscp.net/schema/session/1.0" <session xmlns="http://winscp.net/schema/session/1.0"
-        name="martin@example.com" start="2009-03-02T19:34:57.734Z">+        name="martin@example.com" start="2012-01-04T14:25:53.173Z">
  <group name="put -preservetime -permissions=644 d:\www\about.htm" start="2012-01-04T14:28:45.393Z">   <group name="put -preservetime -permissions=644 d:\www\about.htm" start="2012-01-04T14:28:45.393Z">
    <upload>     <upload>
Line 414: Line 463:
===== [[parse]] Interpreting/Parsing ===== ===== [[parse]] Interpreting/Parsing =====
-To parse the XML log, use any XML parser available. 
-If possible, using [[library|WinSCP .NET assembly]] is preferred approach, which hides away the XML log file from your application.+==== General ==== 
 +To parse the %%XML%% log, use any %%XML%% parser available. 
 + 
 +If possible, using [[library|WinSCP .NET assembly]] is the preferred approach, which hides away the %%XML%% log file from your application.
If you want to parse the XML log yourself anyway, see following guides to learn how to do that in different development environments: If you want to parse the XML log yourself anyway, see following guides to learn how to do that in different development environments:
-  * [[guide_automation_advanced|Guide to advanced scripting]] (Windows script host; Java or VB script); +  * [[guide_automation_advanced|*]] (Windows script host; Java or VB script); 
-  * [[guide_dotnet|Guide for using WinSCP from .NET]] (.NET; C# or VB.NET); +  * [[guide_dotnet|*]] (.NET; C# or VB.NET); 
-  * [[guide_interpreting_xml_log|Guide to interpreting XML log for advanced scripting]] (.NET, C#).+  * [[guide_interpreting_xml_log|*]] (.NET, C#). 
 + 
 +==== [[xslt]] Transforming XML Log to Text Output Using XSLT Transformation ==== 
 +You can use XSLT transformation to convert the %%XML%% log to plain text output (or any other format). 
 + 
 +For example to generate a plain text list of successfully downloaded files from the following %%XML%% log (''log.xml''): 
 + 
 +<code xml> 
 +<?xml version="1.0" encoding="UTF-8"?> 
 +<session xmlns="http://winscp.net/schema/session/1.0" name="user@host" start="2021-12-03T06:45:57.008Z"> 
 +  <download> 
 +    <filename value="/path/file1.txt" /> 
 +    <destination value="C:\path\file1.txt" /> 
 +    <size value="2022" /> 
 +    <result success="true" /> 
 +  </download> 
 +  <download> 
 +    <filename value="/path/file2.txt" /> 
 +    <destination value="C:\path\file2.txt" /> 
 +    <size value="5782" /> 
 +    <result success="true" /> 
 +  </download> 
 +</session> 
 +</code> 
 + 
 +use the following %%XSLT%% (''download.xslt''): 
 + 
 +<code xml> 
 +<?xml version="1.0" encoding="UTF-8"?> 
 +<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:winscp="http://winscp.net/schema/session/1.0"> 
 +    <xsl:output method="text" encoding="UTF-8"/> 
 +    <xsl:strip-space elements="*"/> 
 +    <xsl:template match='winscp:download[winscp:result[@success="true"]]/winscp:filename'> 
 +        <xsl:value-of select="@value"/> 
 +        <xsl:text>&#xa;</xsl:text> 
 +    </xsl:template> 
 +</xsl:stylesheet> 
 +</code> 
 + 
 +You can execute it using any %%XSLT%% processor: 
 + 
 +  * Microsoft ''msxsl.exe'' (deprecated, but [[https://web.archive.org/web/20200202112313/https://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=21714|available from Internet Archive]]): \\ <code> 
 +msxsl.exe log.xml download.xslt 
 +</code> 
 +  * [[http://xmlsoft.org/|Libxml2]] ''xsltproc.exe'': \\ <code> 
 +xsltproc.exe download.xslt log.xml 
 +</code> 
 + 
 +The output will be: 
 + 
 +<code> 
 +/path/file1.txt 
 +/path/file2.txt 
 +</code>
 +For a more complex example, see [[script_custom_listing_format_csv#scripting|*]].

Last modified: by martin

Documentation

Support

Associations

Follow Us