Literate Programming Using FrameMaker

From:     Stephen Cross
Date: 1 Apr 1993
Here is a very quick and simple method for using FrameMaker to write literate programs: 1) Create two conditional_text tags (Special/Conditional Text...) Call the tags something like 'DocText' and 'CodeText'. 2) Apply the CodeText Tag to all the code in the document. 3) Apply the DocText Tag to everything else in the document. 4) To Tangle the document simply change the Show/Hide options for the conditional text so that only the CodeText is shown. 5) Save the Code to a text source file (File/Save As). Okay so its nothing special, but it does work. There are probably other (better) ways of doing this, any ideas?

From:     Lee Wittenberg
Date: 2 Apr 1993
Stephen Cross writes: Here is a very quick and simple method for using FrameMaker to write literate programs: 1) Create two conditional_text tags (Special/Conditional Text...) Call the tags something like 'DocText' and 'CodeText'. 2) Apply the CodeText Tag to all the code in the document. 3) Apply the DocText Tag to everything else in the document. 4) To Tangle the document simply change the Show/Hide options for the conditional text so that only the CodeText is shown. 5) Save the Code to a text source file (File/Save As). Okay so its nothing special, but it does work. There are probably other (better) ways of doing this, any ideas?

It's a nice approach, but it isn't exactly "tangling," as the latter involves rearranging the code chunks from an human ordering (the sections in a web are supposed to be organized in a way that is easy for the human reader to grasp) into a machine ordering (something a compiler can accept). Your approach has the advantage of being able to interleave code and documentation, but it has the disadvantage that the code must still be presented in an order dictated by the machine. (cf. Dijkstra's "... I want the program written down as I can understand it, I want it written down as I would like to explain it to someone." ["Notes on Structured Programming", in "Structured_Programming", Academic Press, 1972]).

From:     Sriram Srinivasan
Date: 25 Oct 1993
I like the concept of literate programming (or what I understand of it). My understanding is: 1) Source and documentation is all present in one document. 2) The same document can be compiled, either for execution by a machine, or for publication. 3) The tools that implement literate programming provide a framework to achieve the above, and macros to format the stuff effectively. Unfortunately, all the examples that I have seen seem to use TeX as a base. I use FrameMaker for all my documents, and using TeX is simply too much effort devoted to formatting. In the current set of tools, one requires a lot of "inband signaling". That is, one has to have special symbols within the document that tag a certain paragraph as a piece of code, or as a part of the body, etc. The resulting document looks really messy, and it's painful to eyeball the document quickly.

I can achieve the same using FrameMaker or most other word processing tools, which allow you to tag paragraph types. So, I can have a paragraph template called "Code", that identifies the Code sections. Now, I can intermix graphics, tables, all kinds of formatting like automatic line numbering, cross-referencing, indexes etc. What I see on the screen is the final output, uncluttered by meta-information. Separating the code from the chaff is very easy, once a Frame file is converted to a MIF file (an awk script is given below). The $(10^6) question: is this idea worth exploring further? I would dearly like your comments on this subject. I am absolutely new to literate programming, and haven't read much beyond the FAQ, and some examples. I would think this can be achieved in say, Word for windows etc. which can save to an RTF format.

For those with access to FrameMaker, this is what I propose:
1. Have a paragraph tag called 'Code', and write all your code in this format.
2. Save as a mif file (or use fmbatch).
3. Run this thru the following awk script 
BEGIN   {
        code = 0
    }

/<PgfTag \`Code\'/    {
        code = 1
        next
    }

/<PgfTag/ {
        code = 0
        next
    }

(code == 1) && /^ *<String \`/{
    # Ignore all other paragraph formats, and get only strings from
    # Code formats.

    # Need to strip this, for example ..
    # <String `printf (\xd2 Hello World\\n\xd3);'>

    # Strip junk in front
    sub(/^[ ]*<String `/,"")

    # Trailing junk
    sub(/'>$/,"")

    # Convert smart quotes to standard quotes
    sub(/\\xd2/,"\"")
    sub(/\\xd3/,"\"")

    print
    next
}
4. If you want pieces of code to go to different files, one can always have a special paragraph type ("File") that identifies a specific file type.

From:     Charles Bass
Date: 25 Oct 1993
I think that your idea with FrameMaker is the only way to go. I looked at a package called ~ winwordweb that was a set of macros for Word for Windows that did what you are doing in Frame. In my opinion this is the nicest way to go because the front end is easy to use WYSIWIG etc. The downside is that it is difficult for others to modify your literate programming program unless the have Frame. The TeX based tools are (fairly) portable across Unix and DOS platforms. At any rate I like your idea of using Frame.

From:     Ralph Johnson
Date: 25 Oct 1993
This is basically what the Literate Programmers Workbench from apple.com does. It is a fine idea.

From:     Lee Wittenberg
Date: 26 Oct 1993
Sriram Srinivasan writes: Unfortunately, all the examples that I have seen seem to use TeX as a base. I use FrameMaker for all my documents, and using TeX is simply too much effort devoted to formatting. In the current set of tools, one requires a lot of "inband signaling". That is, one has to have special symbols within the document that tag a certain paragraph as a piece of code, or as a part of the body, etc. The resulting document looks really messy, and it's painful to eyeball the document quickly.

I can achieve the same using FrameMaker or most other word processing tools, which allow you to tag paragraph types. So, I can have a paragraph template called "Code", that identifies the Code sections. Now, I can intermix graphics, tables, all kinds of formatting like automatic line numbering, cross-referencing, indexes etc. What I see on the screen is the final output, uncluttered by meta-information. Separating the code from the chaff is very easy, once a Frame file is converted to a MIF file (an awk script is given below). The $(10^6) question: is this idea worth exploring further? I would dearly like your comments on this subject. I am absolutely new to literate programming, and haven't read much beyond the FAQ, and some examples. I would think this can be achieved in say, Word for windows etc. which can save to an RTF format.

Actually, what you propose is precisely the way WinWordWEB works, except that it uses internal Word macros instead of saving an RTF file. The idea of a CODE style paragraph that tools can recognize (and tangle and weave, as desired) seems to be the way to go. I recommend you forge on ahead. A lot of people have expressed interest in a FrameMaker-based literate programming tool.

From:     Sven Utcke
Date: 26 Oct 1993
Sriram Srinivasan writes: I like the concept of literate programming (or what I understand of it). My understanding is: 1) Source and documentation is all present in one document. 2) The same document can be compiled, either for execution by a machine, or for publication. 3) The tools that implement literate programming provide a framework to achieve the above, and macros to format the stuff effectively.

Well, my pet concept is missing: the possibility to write the code in any order I could dream off.

I can achieve the same using FrameMaker or most other word processing tools, which allow you to tag paragraph types. So, I can have a paragraph template called "Code", that identifies the Code sections. Now, I can intermix graphics, tables, all kinds of formatting like automatic line numbering, cross-referencing, indexes etc. What I see on the screen is the final output, uncluttered by meta-information. Separating the code from the chaff is very easy, once a Frame file is converted to a MIF file (an awk script is given below)

I certainly like the idea. Although I am unlikely to ever use it myself (I am hooked on LaTeX and wouldn't even consider using one of nowadays WYSIWYG-formatter --- the output is much to ugly), it seems to be a good way to introduce more people to the concept of literate programming. If only the "chunk"-concept would be used (but that can't be so difficult). On the other hand: Isn't that what CLiP is doing?

From:     Bob Collins
Date: 27 Oct 1993
Sriram Srinivasan recommends using FrameMaker for literate programming. However, Sriram Srinivasan did not mention all the advantages of FrameMaker. FrameMaker is, among other things, a hypertext document generator. One can construct (semi-automatically in FrameMaker 4) a set of hypertext links so that clicking on a hypertext word or picture will display related text. One can have the effect of a table of contents or index without page numbers. One can click on the "title" of a related code fragment to display the code fragment. Consider the primary advantage of a hypertext WYSIWYG document. One is always working with the latest version of the final document. No wasting of printer resources or working from slightly outdated printed documents.

FrameMaker also has an easy to use book concept with multiple chapters. This is the most logical way to break up a large document. FrameMaker will maintain and regenerate all sorts of links across documents. No need to write obscure macros. No need to modify these macros. FrameMaker allows all sorts of tagging (invisible to the eye but denoted in the bottom margin) of paragraphs and text. Uses of these can be so that lists can be generated and text extracted. This is how one can an automatic cross-reference and make it a hypertext cross-reference. There will no need to use cryptic marks in the text to indicate special things (like the continuation of the body of a code fragment. Use tags and a special typographic standard. Use variables to say "continued." All of these are easily feasible.

Sven Utcke objects that he would no longer be able to write code in an arbitrary fashion. Well, he may have misunderstood Sriram Srinivasan who claimed that with almost no effort one can use FrameMaker for literate programming. [Indeed, using conditional text for code, within FrameMaker one can view just the code using a simple dialog box. This requires no work at all.] Don't forget that the Knuth system involves running programs to put code fragments together in the correct order. One can also write a program to do a similar thing to MIF output of FrameMaker documents. MIF is Maker Interchange Format, a TeX- or RTF- like marked ASCII text.

I have been reading this group for some time. I agree with all those who say the biggest drawback to literate programming is learning a new language -- TeX. TeX, written in the late 70's and popularized in the 80's is very much a 60's product in its interface. Granted it does things that were not done in the 60's. But it sure feels like JCL. Knuth, a wise and witty man, is a hacker at heart. He thinks obscurely and at a low level. He chooses low-level assembly language to describe algorithms. I cannot figure out how he has such a delightful sense of humor and an assembly language mentality. We seem to apotheosize the one person who least practices the things that make literate programming such a nice idea. [Knuth reminds me of a story (apocryphal?) about John von Neumann. At a party von Neumann was asked the puzzle: two trains 100 miles apart are approaching each other at 50 mph. A bumblebee, starting at one train, travels between trains at 25 mph, reversing directions when reaching an approaching train. How far does the bumblebee travel before being crushed? von Neumann thought a short moment and promptly answered 25 miles. When asked if he knew the trick, he acted puzzled and said that he summed the infinite series.]

The only advantages of TeX that I know of are that it does mathematical typesetting nicely and it's free. FrameMaker does mathematical typesetting sufficiently well for literate programming.

From:     Sriram Srinivasan
Date: 27 Oct 1993
Bob Collins writes: Sriram Srinivasan recommends using FrameMaker for literate programming. However, Sriram Srinivasan did not mention all the advantages of FrameMaker. FrameMaker is, among other things, a hypertext document generator. One can construct (semi-automatically in FrameMaker 4) a set of hypertext links so that clicking on a hypertext word or picture will display related text. One can have the effect of a table of contents or index without page numbers. One can click on the "title" of a related code fragment to display the code fragment.

[ FrameMaker plug deleted ....] I have been reading this group for some time. I agree with all those who say the biggest drawback to literate programming is learning a new language -- TeX. TeX, written in the late 70's and popularized in the 80's is very much a 60's product in its interface.

You can say this again!

Now, for the downside of using Frame, or any other word processor. Note that I am definitely not in favor of TeX still. 1) Speed of development - I am a power Emacs/Vi user, and Frame or WFW simply doesn't cut it in terms of development speed. 2) All my text processing tools can't be used - egrep, class browsers, awk, sed, sdiff, tags, and a zillion others. I managed to fix my mifToCode script to generate the correct line numbers, so that GDB isn't confused - but you see the point - I now have to worry about issues that weren't in text-land. 3) SCCS isn't very happy either. Have to move my stuff over to RCS, to be able to check in or out stuff. Either that, or have to convert all files to MIF before using SCCS. 4) Integrating this with other code in a production development environment seems to be a pain (I tried). (Other code - stuff already developed). 5) Using TeX and remaining in text-mode is a concept I find very painful. I do not want to think formatting when I am into development. Bob Collins has already said all I would have wanted to.

I guess I need a powerful argument for undergoing these hassles, especially the development cycle speed, before I really change my ways. Practically, how does it payback? Does it result in fewer edit/compile/run cycles? Has anyone used it for projects that have, say, 2 million lines of code, and has to be pushed out, say, yesterday? (Someone other than Knuth, that is!) Meanwhile, I convinced one of my friends to use my scripts and paragraph formats for a book he's working on. At the very least, we can test out the code that is going into the book by extracting all of it and compiling it. He's not completely in favor of doing development inside Frame yet.

From:     Joachim Schrod
Date: 28 Oct 1993
Bob Collins writes: Knuth, a wise and witty man, is a hacker at heart. He thinks obscurely and at a low level. He chooses low-level assembly language to describe algorithms. I cannot figure out how he has such a delightful sense of humor and an assembly language mentality.

How? Simple: he's a scientist. How do you do exact analysis of run-time behavior without an exact notion of the cost of basic operations? Give me the answer, and then we can see if we can get rid of MIX. BTW, Vol. 4 won't have MIX any more.

The only advantages of TeX that I know of are that it does mathematical typesetting nicely and it's free. FrameMaker does mathematical typesetting sufficiently well for literate programming.

How do you edit your FrameMaker documents on more than 40 different operating systems, ranging from 8086 PCs to Crays? How do you use FrameMaker in a convenient way as a background system? How do you use many languages? (Remember: Not everyone writes in English...) Don't get me wrong: FrameMaker is surely one of the better commercial systems around -- but your evaluation of the differences between TeX and FrameMaker is not even.

From:     Mark Friedman
Date: 28 Oct 1993
From Joachim's response to Bob Collins' comments on Framemaker's strengths: How do you edit your FrameMaker documents on more than 40 different operating systems, ranging from 8086 PCs to Crays? How do you use FrameMaker in a convenient way as a background system? How do you use many languages? (Remember: Not everyone writes in English...)

I will add 2 even "lower-end" systems to the editing list --- I often take notes & revise drafts on an ancient Tandy T102 (not even MS-DOS, but it can spit out ASCII), and I have 3 daughters (ages 11 to 18) that "share" an 80286 to (La)TeX documents created on a trio of (also ancient) Apple ]['s. The point, of course is quite simple --- if it can produce ASCII output, anything can be used for entry & editing.

As for the oft cited difficulty of learning LaTeX, my daughters have done all their school reports since 3rd grade as well as lots of other "documents" (my 15 year old produces Girl Scout troop meeting minutes, plans & budgets), in English, French, German, Spanish and even ASL (American Sign Language --- a one-off report using a wonderful `hands' METAFONT file). Overkill? Absolutely, but they happen to love it (they're even more critical of sloppy or ugly documents that I am, and I thought I was the perfectionist in the family). It has also led to various amusing incidents, like when my oldest was in 7th grade her English teacher stated that there was "only one right way to format a bibliography". Allyson's response was to take my copy of the Chicago Manual of Style to school, along with about a dozen versions of her assignment's bibliography, all formatted by different .bst styles, and ask which one was the "right" one...

From:     Bob Collins
Date: 29 Oct 1993
This is my one response to my complaint about TeX and Knuth. I have the feeling this might turn into a jihad. FrameMaker is a commercial product suitable for producing large printed and hypertext documents, especially those that need maintenance and involve technical material and drawings. FrameMaker cannot do all things and some of what it does it does poorly. FrameMaker is not even my word processor of choice, but I use it because of its universality.

Joachim Schrod wrote: How do you edit your FrameMaker documents on more than 40 different operating systems, ranging from 8086 PCs to Crays?

FrameMaker is available on all the major platforms that can support it, that is, that have sufficient power and memory and that allow bitmap displays. This means most Unix systems, Macs, and MS Windows. Don't blame FrameMaker if it doesn't work on an AT. One cannot have its power for free. TeX is a filter -- all it does is translate from one form to another, like a compiler. Therefore it needs far fewer resources. FrameMaker integrates what TeX does, what your favorite editor does, what previewers do, what drawing programs do, and what dvi2lp utilities do. Face it -- how many of you wished that you could edit your preview when you found some error that you overlooked? How many of you preview on an 8086?

Joachim Schrod wrote: How do you use FrameMaker in a convenient way as a background system?

FrameMaker (on Unix) has both a macro system and a batch system. Both allow background processing. [In a silly moment I might ask, how do you allow foreground processing of TeX commands while you edit?]

Joachim Schrod wrote: How do you use many languages? (Remember: Not everyone writes in English...)

Last I checked, FrameMaker is available in 11 languages. (12, if you separate American English and Proper English.) This support is available in the dictionaries, thesaurus, system variables (like dates), and numbering schemes.

Mark Friedman wrote: I will add 2 even "lower-end" systems to the editing list --- I often take notes & revise drafts on an ancient Tandy T102 (not even MS-DOS, but it can spit out ASCII), and I have 3 daughters (ages 11 to 18) that "share" an 80286 to (La)TeX documents created on a trio of (also ancient) Apple ]['s. The point, of course is quite simple --- if it can produce ASCII output, anything can be used for entry & editing.

One can import text into FrameMaker. FrameMaker will even get rid of extraneous spaces and turn straight quote marks into curly ones while it imports (if you want). In addition, FrameMaker supports MML import. MML stands for Maker Mark-up Language and is a SGML-like form for typographic conventions. So your kids can edit their instructions in their text, read it into FrameMaker, and then edit the preview-like display.

Mark Friedman wrote: As for the oft cited difficulty of learning LaTeX, my daughters have done all their school reports since 3rd grade as well as lots of other "documents" (my 15 year old produces Girl Scout troop meeting minutes, plans & budgets), in English, French, German, Spanish and even ASL (American Sign Language --- a one-off report using a wonderful `hands' METAFONT file). Overkill? Absolutely, but they happen to love it (they're even more critical of sloppy or ugly documents that I am, and I thought I was the perfectionist in the family). It has also led to various amusing incidents, like when my oldest was in 7th grade her English teacher stated that there was "only one right way to format a bibliography". Allyson's response was to take my copy of the Chicago Manual of Style to school, along with about a dozen versions of her assignment's bibliography, all formatted by different .bst styles, and ask which one was the "right" one...

Well, Mr. Friedman has a smart set of kids. I congratulate them and him. I have my own set of computer-literate kid stories, and I will be pleased to share them. I wish all children had entire to computer-wise parents and equipment at the earliest age. It would help me as a university professor. [My favorite kid story is how my son, when 5, learned how to use a paint program by memorizing what the menus did. He could not read, but he knew the alphabet.]

However, I have gone through the hassle of trying to help graduate students try to write using LaTeX. The learning curve is steep. In fact, some of our faculty couldn't figure out how to get rid of a space before a grammar production in our comprehensive examination question list. The number of macros (written by different folks) confused the issue. The smartest people can get turned off by TeX. Is literate programming closed to them?

Let's get serious, however. For those who know, TeX works great. Just like punch cards for editing. However, most people found interactive editing much easier. And those who were used to punch cards sometimes found it difficult or unnecessary to change. And they were right. You can do anything you want with a punch card. Make a wish list for literate programming. Don't worry about how it can be accomplished, or how the typesetting is done. I think that most of us would like to edit the finished product rather than the marked-up product. [I would like to do it by voice, e.g., Bob Collins: Show me the code fragment for the input loop. Computer: Did you mean code fragment or chunk?] At the least, this means interactive TeX. Let's aim for that. Meanwhile, I think that FrameMaker offers a reasonable alternative. These are my final words. Flame on.

From:     Jim Rudolf
Date: 29 Oct 1993
Charles Bass writes: I think that your idea with FrameMaker is the only way to go.

Personally, I think using FrameMaker is a good idea, but not necessarily by editing in FrameMaker. We are just a bunch of programmers who use vi and work fast in it. We want to be able to document our code too, and embedding the doc in the source code (keeping in mind the readability of the source for us programmers, of course) means we are more likely to update the doc when we update the code. Our solution was to write a couple Perl scripts to do documentation generation for us. One script scans the source file for a few keywords that can describe the entire module or individual procedures. Once all the info for a file is collected, another set of scripts will create a file either in man page or MML format. The two formats are quite different, as the man page format is used internally, and the MML format is used to make a FrameMaker book for customers. Other back-end scripts could easily be written when needed. We've found this to be a good combination for us. Has anyone else had any experience (good or otherwise) with this approach?

From:     Michael Koopman
Date: 29 Oct 1993
Bob Collins wrote: This is my one response to my complaint about TeX and Knuth. I have the feeling this might turn into a jihad. FrameMaker is a commercial product suitable for producing large printed and hypertext documents, especially those that need maintenance and involve technical material and drawings. FrameMaker cannot do all things and some of what it does it does poorly. FrameMaker is not even my word processor of choice, but I use it because of its universality.

I don't believe you'll see an uprising from your comments on TeX. I have not heard anyone pining over the punch card machine nor against an interactive TeX editor/previewer. The TeX hacks promote the capability, portability and succinctness of the language. Whether or not FrameMaker is as clever as TeX seems somewhat irrelevant. If you want a FrameMaker web then what qualities of the web are critical and which of these 'requirements' are met by FrameMaker and not met by TeX. That is, why does literate programming in FrameMaker detract from literate programming in TeX? Does FrameMaker read TeX input, or more importantly, can FrameMaker read a CWEB, FWEB, etc. web and not munge it beyond recognition? Let's not hack each other up in religious wars over the "right" editor or typesetting mechanism. Let's focus on the benefits and needed improvements in the literate programming techniques at hand, instead. Peace be with the Literati.

From:     Osman Buyukisik
Date: 29 Oct 1993
People seems to be missing the main point about literate programming. It is not just a documentation scheme! You start from the design phase and use whatever form of editor/TP you want to explain the code and write the code in a way that is independent of the whims of the compiler that you will use. In the end, the final product should resemble a `book'. Because of these points I don't think c2man and others like it are literate programming tools. You can use FrameMaker/interleaf but someone has to write some expansion to it so that `tangle' command is available in the menu (like the WinWordWEB). No one is forcing TeX/LaTeX but these tools are FREE and widely available. People should not be flaming each other instead if you want a FrameMaker literate programming tool just create it and announce it to the net. For myself I cannot see how hypertext helps since I like the printed output! I use interleaf at work and all the formulas and graphs do not look the same on the screen as printed on paper, and are hard on the eyes.

From:     Uttam Narsu
Date: 7 Apr 1994
Does anyone know of any Web (literate programming) tools for FrameMaker? We need to be able to keep mathematical equations, graphics and code together. I have taken a look through the FAQ, but didn't see anything that would really help us. Most Web tools won't be acceptable to our programmers because the tools demand some knowledge of TeX (especially for formatting equations and pictures). Since Frame is already used for much of our documentation, it seems to be a natural choice for a front-end to a Web.

The FAQ mentions two tools that would almost be useful: CLiP and WordWeb. Alas, CLiP is written in ISO Pascal, which is useless to me since I don't have a Pascal compiler for my SGI. WordWeb is completely dependent on Microsoft Word, so no help there. As a last resort, if anyone knows of any filters between TeX and Frame MIF files, that would be helpful.

From:     Jacob Nielsen
Date: 7 Apr 1994
Does anyone know of any Web (literate programming) tools for FrameMaker?

Some time back, there was someone who had done literate programming using FrameMaker, anyone got that posting?. In the meantime take a look at nuweb. You can tell nuweb to only produce the code files and not output the documentation. The downside is that you have to look at scraps like this in the FrameMaker document: 

@o test.c 
@{
  @<Include files for the test@>
  @<Defines for the test@>
  @<All the rest...@>
@}

If you can save the FrameMaker document, so that you have text like this, you should be home free using nuweb.

From:     Sriram Srinivasan
Date: 9 Apr 1994
For those with access to FrameMaker, this is what I propose:
1. Have a paragraph tag called 'Code', and write all your code in this format.
2. Save as a mif file (or use fmbatch).
3. Run this thru the following awk script. 
BEGIN   {
        code = 0
    }

/<PgfTag \`Code\'/  {
        code = 1
        next
    }

/<PgfTag/ {
        code = 0
        next
    }

(code == 1) && /^ *<String \`/{
        # Ignore all other paragraph formats, and get only strings from
        # Code formats.

        # Need to strip this, for example ..
        # <String `printf (\xd2 Hello World\\n\xd3);'>

        # Strip junk in front
        sub(/^[ ]*<String `/,"")

        # Trailing junk
        sub(/'>$/,"")

        # Convert smart quotes to standard quotes
        sub(/\\xd2/,"\"")
        sub(/\\xd3/,"\"")

        print
        next
    }

4. If you want pieces of code to go to different files, one can always have a special paragraph type ("File") that identifies a specific file type.

From:     William Clodius
Date: 18 Oct 1994
I am interested in the potential for literate programming, but most literate programming tools appear to rely upon TeX or LaTeX. The team I am involved with are doing most of their documentation with FrameMaker. Does CLiP or some other tool facilitate literate programming with FrameMaker?

From:     Eric van Ammers
Date: 25 Oct 1994
Regarding your mail of on the question if CLiP can be used with Frame Maker, I expect no problem. CLiP can be used in combination with any system which provides for a decent ASCII-export.