Posting Large Binaries to Usenet
version 0.1 April 2003
- a work in progress
Defining the Terms
Segment
Part
Clip, Set, or File
Post
Planing Out the Post
The Subject Line Header
Things that go wrong when Placed in the Header
Verification and Recovery Tools


Defining the Terms used here:
Here's a sample of a typical large binary post Header; first from Agent:


and another from XNews:


1. Segment:  the line count or byte size of each message posted. In the examples above, you don't see the segments on the complete post, only the indicators of the number of segments - in Agent it's the (*/81), in Xnews it's the [~~/81]. If you're using a low grade reader, or if you change the view to see each separate segment as it was posted to usenet, it would look like this in Agent:

and in Xnews:


note the change form (*/81) to (01/82)-(02/82), etc. Each of these "01,02" denotes a segment. In this case the segment size shows a line count of 4162 lines. While we'll get into it later, for now just know that this was posted in UUE, and 4162 lines is around a segment size of about 1/4 of a meg, or 256kb (no magic, I downloaded one to find out). Each of these is an individual usenet message with it's own id, header, content information, and reference (where it belongs within the rest of the messages of it's part). If you don't understand why messages are done this way on usenet, start by reading this information.

2. Part:  In the case above, a Part is the 82 separate usenet messages joined together. Once uploaded to a usenet server, and downloaded to other's computers, the 82 parts would provide you with  a complete message with about a 20 meg file size. In many cases, this does not mean that you would have a usable clip. With the above sample. the "part14.rar"  would indicate that you would have at least 14 files (part01.rar to part14.rar) before you could move forward and have a complete standalone useable post.

3. Clip, Set, or File:  While three terms will changed based on the content of your  posted messages, they all refer to the "if I get just this much, I'll have a usable something". That something may be a view-able video or audio clip, a set of pictures zipped or Rared together (a single picture would be complete in and of itself, half a movie, or a cd of data. It would also include support files, including nfo, preview(s), links, sfv, and Par files that directly refer to this single group of items.


4. Post: The complete group of files posted by a single poster on a given topic or theme. This could be the same as #3, a set of clips, or it could be that it would take a series of Sets to complete the Post - part 1, 2, and 3, of a movie posted with separate par  and sfv's for each part; or a 3 cd set of your collection of toenail fungus, each again with it's own support files, would also qualify.



Planning out the Post
Dealing with any large binary post, wether downloading or uploading, requires some consideration. How many parts to the set? How large a post? What limits to uploading will I encounter - speed, volume, propagation (if you don't understand the importance of propagation, then you skipped over reading Usenet. Go back and read it). If you plan on posting video that you've capped yourself, you should have read Capping and Editing first. (You did? Looks to me like you just skimmed over it. Read it. The post won't burn a hole on your hard drive while you do.)
Planing the post involves writing an introduction - a note that describes the contents of your post, the rate of your posting, and players/codec/support files included; creating support files - Par files to allow the downloader to confirm uncorrupted files and repair them if corrupt, sfv files that allow for confirmation but no repair of the file; and writing a header - that subject line that gives the first hint to a downloader about your post.
Having written the steps above backward, we will now do the steps in the order they are best done in.

    The Header:
The Subject line of a post has affected it's acceptance more than any other action a poster can make. Forgeting that " " spaces in the files title will cause most servers to rename the file, mostly from "word nextword" to "word_nextword" on more than half of all servers. no biggie until you see how it impacts your sfv and par files. Placing "/" slashes in headers will be read by most news readers as a part count, and it will attempt to assemble them into one post. Mostly on readers like Agent, your post will just look like an incomplete post.
Here are a list of common "don'ts" - things to leave out of your header and title:
Issues with the use of certain characters in the header. One is the problem with drn (web based usenet access) that alter " "(space),"&"(the and sign) and so on which are not valid within the DRN/HTML file naming system. You might want to check http://www.december.com/html/spec/esccodes.html for a fairly complete list.
The other seems to be issues mainly with Agent's handling of filenames - The same feature that joins multi-segmented files into one header seems to trip up Agent. Avoid the use of the '/', '\', '#" and ' " ' characters in the description.

Problem characters:

No spaces are permitted.
there are these that will cause trouble with web/drn servers: These will cause trouble with either drn servers or
PP2K (sending) & Agent (downloading).
the left angle bracket (<),
the right angle bracket (>),
the ampersand (&)
the Double Quote (")
the forward slash (/)
the back slash (\)
the 'at' (@)
the dollar sign ($)
the Piping symbol (|)
the Colon (:)
the ' quote as in Amanda's
the * asterick
the ? question mark
the < less than cosign
the > greater than cosign
For example, the following header
NEW file 1/4 - nifty_file.mpg(01/16)
Makes Agent look for four parts of a file named nifty_file.mpg(01/16). Instead, use NEW '1 of 4', '1 - 4', or simply '4-parts'  (thanks PunkKid and Solly) "Jonnie does Debbie"can also throw off readers like Drn/webservers (spaces). Instead, use 'Jonnie-does-Debbie'.  And many newsreaders react to # - the pound sign in a multi part post - best to skip them all together.

    Verfication and recovery files: Quite simply, for any post that contains more than 5 10 meg or greater parts, you will be doing yourself and others a great favor by creating a means by which they may verify and repair any badly downloaded parts. Why? Because of the nature of usenet, and the nature of lurker / downloaders.  First, let's deal with verification:
            SFV files: the road to understanding file download failure.




Posting order
    1. NFO - text file with info on post
    2. Preview  - small sample file (either jpg or small clip) showing action and most importantly - qualitiy. Keep it to NO MORE than a couple of stills per clip, or  1 per min of flim.
    3. Par / sfv - the root Par 1, Par 2, or Sfv file so that the downloader can check the download as he gets it.
    4. The bulk file(s)
    5. The Recovery Files. Enough is no more than 20% of the wait of the clip. If Par two, make the "block size" equal to your posting segment size.
If a multiday post, repeat Nfo, Par, and have previews for each day's post.