Including Token-Delimited Files

A common function of many dynamic web sites is to post the contents of token-delimited files (such as Linux Today's headlines file) on their web site in some kind of programmatically filtered format. These filters generally are implemented differently from page to page, and site to site, and rely on somewhat involved algorithms to pull apart the data and put it back together again into a useful format.

The LXP approach to displaying such files is with the use of the <include> tag, by specifying the method="parsed" attribute. This use of the <include> tag breaks up the parsed fields into sequential values, accessible via the general-purpose LXP <field> tag.

Blocks are delimited from one another by the value supplied to the delimiter attribute. Within a block, fields are separated from one another by each newline (symbolically, \n, a literal line-wrap) found within the block. You may optionally specify a different field delimiter value using the separator attribute.

The parsed method for the <include> tag requires a closing </include> tag, because for each block that LXP reads from the file, it loops back to the beginning of the <include> tag and re-iterates the mark-up until the last block is processed.

If you wish to limit the number of blocks to be displayed, the last block number can be specified with the lastblock attribute. Additionally, the firstblock attribute can be used to skip any leading blocks (e.g., an introductory statement that might be embedded at the top of the text file preceding the first delimiter).

Here is an example of such a token-delimited file, from www.linuxtoday.com :

Welcome to lthead.txt.  Fields are delimited by two ampersands.
The first field is the headline.  The second field is the URL to the story.
The third field is the date the story was posted.
Have Fun! ([email protected])
&&
LinuxProgramming: python-dev summary 2001-06-21 - 2001-07-05
https://linuxtoday.com/news_story.php3?ltsn=2001-07-05-019-21-OS-SW
Jul 5, 2001, 21:30:38
&&
Chicago Sun-Times: Test drive Linux using friendly tryout software
https://linuxtoday.com/news_story.php3?ltsn=2001-07-05-018-21-PS-CY
Jul 5, 2001, 21:00:48
&&
[...]

Example 13-23 opens the file /home/web/headlines/lthead.txt , and parses it into blocks using the && character sequence as the block delimiter.

<lxp>
  <include src="/home/web/headlines/lthead.txt" delimiter="&&"
           firstblock="2" lastblock="4" method="parsed">
    <table border="0" cellspacing="1"><tr>
      <td bgcolor="#ffffff" width="100%">
        <div class="content">
         - <field />
        </div>
      </td>
    </tr><tr>
      <td bgcolor="#e0e0e8" width="100%">
        <strong>
        <field type="url" link="Read More..." target="_blank" />
        </strong><br />
      </td>
    </tr></table>
  </include>
</lxp>

When an inclusion such as the one in Example 13-23 is processed, the <field> tags are replaced with the field values found within the parsed blocks. Fields are assigned to <field> tags in the order in which they are found.

As you can see in Example 13-23, you may also specify an alternate type attribute for an LXP <field>. Valid types in a parsed inclusion are hidden (this hides the field if there is a value that you wish to skip over, and not display) and url.

The hidden type is used for a field which you wish to merely skip over. Since token-delimited files have no identifying name for each block, each field must be processed in the order that is encountered by LXP in the source file. Therefore, a field can be assigned a type="hidden" attribute in order to skip it rather than display it, allowing you to display fields that are past it in the file.

The url type is useful in this context when you know that a particular field will be a URL, as it creates a hyperlink to that URL (with an HTML <a> tag), rather than just displaying the URL itself. You can set the text of the generated hyperlink to appear as an arbitrary value, other than just the URL itself (such as the Read More... value used in Example 13-23), by specifying the value of the link attribute within the <field> tag.

Here is example output of what you would see from LXP, after parsing the mark-up from Example 13-23:

<table border="0" cellspacing="1"><tr>
  <td bgcolor="#ffffff" width="100%">
    <div class="content">
     - LinuxProgramming: python-dev summary 2001-06-21 - 2001-07-05
    </div>
  </td>
</tr><tr>
  <td bgcolor="#e0e0e8" width="100%">
    <strong>
    <a href="https://linuxtoday.com/news_story.php3?ltsn=2001-07-05-019-21-OS-SW"
target="_blank">Read More...</a>
    </strong><br />
  </td>
</tr></table>

<table border="0" cellspacing="1"><tr>
  <td bgcolor="#ffffff" width="100%">
    <div class="content">
     - Chicago Sun-Times: Test drive Linux using friendly tryout software
    </div>
  </td>
</tr><tr>
  <td bgcolor="#e0e0e8" width="100%">
    <strong>
    <a href="https://linuxtoday.com/news_story.php3?ltsn=2001-07-05-018-21-PS-CY"
target="_blank">Read More...</a>
    </strong><br />
  </td>
</tr></table>

[...]