On the other hand, storing data in main memory also creates some difficulties. For example, all data must be provided to the program each time the program is run, and the amount of data is limited by the size of main memory. In addition, except for information specified in define statements, any data passed through parameters or computed during procedure execution are destroyed after each procedure is completed executing, and data are not stored from one use of Scheme to the next.
This and the next several labs introduce files as a mechanism to overcome some of these desadvantages. Specifically, disk files provide a way to store information in bulk over a long period of time. In applications, we may give up some efficiency, as data on files normally must be read into main memory before the data can be used. Similarly, any results we want stored permanently must be written out explicitly to a disk file. While this storing and retrieving of data requires some specific work, the use of files does overcome the disadvantages mentioned above.
Since the storage of data within files is so important, at least three perspectives have evolved for file processing:
streams: files are considered to contain a sequence or stream of data, such as numbers or characters,
text files: data within files are considered to be organized line-by-line, and
random-access files: data are placed in files, to be accessed in any order.
This lab introduces file streams, which are viewed as sequences of data; the next lab discusses text files. Random-access files constitute a more advanced topic and are beyond the scope of this course.
To introduce some basic ideas of streams, this lab begins by developing solutions for three problems. In each case, we assume that data have been placed in a file using an editor (such as xemacs). That is, we assume that we have typed data into an editor and saved the data in a file which we have named. As with other files we edit, we could go back to our editor to revise or expand the data whenever we wanted.
For the problems that follow, the lab first describes a solution by using pseudocode -- a step-by-step outline of what we need to do to solve the problem. Then, the lab develops the Scheme implementations of those solutions.
Problem 1: Find the sum of the numbers in a given file.
Solution Outline: The general form for this problem follows a common approach for much file processing. First, we open the file; then, we read and process data; finally, we close the file and print the results. The pseudocode solution that follows adds a few more details:
Solution in Scheme: Here are the built-in Scheme procedures, together with some programming hints, that can help us do various parts of this job:
open-input-file procedure takes as argument a string
that names an existing, accessible file; it returns a ``port'' through
which data values, such as integers, can be read in from the file.
let expression.
let initializes the running total
to 0 and initializes the parameter next as the first value read
from the file.
read procedure, which we have previously seen in its
zero-argument form as a way of obtaining data from the user at the
keyboard, also has a one-argument form: If an input port is supplied as the
argument read reads in, through that port, the next value
stored in the file.
read reports
this fact by returning a special ``end-of-file object'' instead of a normal
datum. The eof-object? predicate takes one argument,
typically something that read has just returned, and itself
returns #t if its argument is the end-of-file object and
#f otherwise.
close-input-port procedure takes one argument, an
input port, and closes it, freeing the resources used to connect the
program to the file.
(define sum-of-file
(lambda (source-file-name)
;Pre-condition: source-file-name is the logical name of a file of numbers
;Post-condition: returns sum of numbers in the given file
(let ((source (open-input-file source-file-name)))
; Open the file.
(let loop ((total 0) ; Initialize the running total.
(next (read source))) ; Try to read a number.
(if (eof-object? next) ; If you get the end-of-file object,
(begin
(close-input-port source) ; close the file
total) ; and report the final total.
(loop (+ next total) ; Otherwise, add the number to
; the running total,
(read source))))))) ; try to read another number,
; and repeat the loop.
A typical interaction using this procedure would look like this:
> (sum-of-file "/home/walker/151s/labs/file1.dat") 200The file
/home/walker/151s/labs/file1.dat contains the
four numbers
50 50 75 25
(read source) appears twice in the
above code. What is the purpose of each appearance of this expression?
In reviewing this processing, note that the data file was viewed as containing a sequence or stream of numbers. Processing data in this file then involved reading the numbers, value-by-value, until we reached the end of the file.
Assume that a sentence is any sequence of characters ending with a period, question mark, or exclamation point. Modify tally-char, to get a procedure count-sentences, which determines the number of sentences in a file.
File /home/walker/151s/labs/lab-file-description contains the first paragraph of introductory material in this lab, starting "Up to this point, ...". Check that this paragraph contains 2 sentences:
(count-sentences "/home/walker/151s/labs/lab-file-description")
===> 2
Line breaks in the input file should be ignored. In the output file, arrange for each integer to be printed on a line by itself.
The following problem may be useful in estimating the length of a paper, as one rule of thumb states that a typical page of double spaced manuscript contains about 12 words per line and about 27 usable lines (excluding margins) for a total of about 324 words.
Problem 3: Approximate the number of words in a file.
Solution Outline: To approach this problem, we need a working definition of a "word". One possibility is that a word is any sequence of character that begins with a letter and ends with whitespace (e.g., a space, a tab, or a newline).
To count words, we read a file one character at a time. When we first encounter a letter, we have another word. We then skip over additional letters (or non-white space) as part of that word. One possible outline for such a solution follows:
Solution in Scheme: Since the finding the start or the end of a word requires checking various details, we write steps 2A and 2Bii as separate, mutually-recursive, local procedures. Thus, find-start-word reads character-by-character until finding the start of the word (step 2A), while find-end-word reads character-by-character until the end of the current word (step 2Bii). All of step 2 is accomplished by having these two procedures call each other. The resulting code follows:
(define count-words
(lambda (source-file-name)
;Pre-condition: source-file-name is the logical name of a file
;Post-condition: returns number of words in the given file
(letrec ((source (open-input-file source-file-name))
(print-result
(lambda (count)
(display "File ")
(display source-file-name)
(display " contains ")
(display count)
(display " words.")
(newline)
))
(find-start-word
(lambda (next-char count)
(cond ((eof-object? next-char)
(begin
(close-input-port source)
(print-result count)
))
((char-alphabetic? next-char)
(find-end-word (read-char source) (+ 1 count)))
(else
(find-start-word (read-char source) count))
)
))
(find-end-word
(lambda (next-char count)
(cond ((eof-object? next-char)
(begin
(close-input-port source)
(print-result count)
))
((char-whitespace? next-char)
(find-start-word (read-char source) count))
(else
(find-end-word (read-char source) count))
)
))
)
(find-start-word (read-char source) 0)
)
)
)
Check that this program works by running it on the file /home/walker/151s/labs/lab-file-description which contains the first paragraph of introductory material in this lab, starting "Up to this point, ...". (The paragraph contains 44 words.)
Describe in several sentences how this program works.
count-words is incomplete, in that it contains few comments beyond pre- and post-conditions. Add appropriate commentary to clarify the purpose of each of main part of the code.
Modify count-words so that a word is considered to be only a sequence of letters. That is, for this part, a word is a sequence of letters -- without punctuation or digits.
Challenge Problem: Use the ideas of count-words and count-sentences to write procedure average-words, which determines the average number of words in a sentence. Note that for efficiency, average-words should only read through the file once.
This document is available on the World Wide Web as
http://www.math.grin.edu/~walker/courses/153.sp00/lab-file-intro.html