From: mash@mash.engr.sgi.com (John R. Mashey)
Newsgroups: comp.arch
Subject: Re: Ultrasparc 1 has no 64 bit pointers?
Date: 30 Apr 1996 00:51:57 GMT

In article <4m2n1n$36i@ansible.bbt.com>, bnm@indica.bbt.com (Brian N.
Miller) writes:

|> Bottom line:  Memory mapped file access is almost always more efficient
|> for system throughput than explicitly managed file buffers.

In general, I think Memory-Mapped files are A Good Thing to have ... but the
generalization above needs more actual data to support it...
"almost always" seems a little strong, given that:

	(a) RDBMS do a whole lot of explicit management of files, even when
	they have file-mapping of huge files available.  Maybe this is
	historical, or maybe there are good reasons for doing this.
	Oracle certainly manges its own LSGA itself, and has no desire for
	the OS to page things in and out of there...
	Is the assertion above that they are wrong to be doing this?
	(I.e., are RDBMS considered covered in the domain of discourse,
	or not?)

	(b) While I have no direct experience, I'm told by people who know
	that various kinds of image processing or visualization programs do
	explicit management of their file data, (which doesn't fit in
	memory).  They do this to get smooth scanning of detailed images,
	or smooth flyovers of terrain (such as image/geometry
	combinations). The only way they are able to do this is to
	"read-ahead" in the database in the direction they are going at the
	moment.  This is bad enough for 2-D pan-and-zoom, but of course, is
	worse for 3D flyovers.  It is, of course, rather difficult for an
	OS to outguess some application-orinted file format that has a
	mixture of 3D geometry and satellite images, and that in fact, is
	doing direction-prediction in response to a joystick.

	(c)  Various systems that support mapped files also support
	asynch I/O to let user programs manage their own buffering.
	Maybe they need to do this, maybe they don't ... but customers demand
	it.

Anyway, I'm not sure that categorizations like "almost always" help gain
much insight, especially as "almost always" must mean "almost all of the
members of some set of occurrences", and it is not all clear what that set is.
It might be more useful to explore the attributes that favor or disfavor
file-mapping compared to explicit management, also learning from some
past history (like explicit overlays of code versus paging.)

+ Favor file mapping
P1	OS has global idea of memory availability, i.e., a specific
	application that manages memory explicitly will tend to be less
	dynamic in its adaptability.
P2	Maybe the OS has a better set of algorithms.
P3	Some code may be simplified, especially that which randomly accesses
	a bunch of pages, in ways that have no particular predictability
	in location or size.  [One an imagine, for example, that you had a
	set of files representing a Boeing 777, and you wanted a browser that
	could look at the whole thing, or look at a doorknob, or anything
	in between, and ran on anything from a desktop to an 8GB Onyx.]

- Disfavor file mapping
N1	Consistency/performance issues: DBMS do *not* want an OS's guesses
	to control the timing of updating files.  They do *not* want an
	OS to write a disk block out just because the DBMS wrote 4 bytes
	there.  They do *not* want an OS to arbitrarily defer an update..
N2	The application knows something about the reference patterns that
	would be extremely difficult to tell an OS or for the OS to infer.

Now, for paging of code, people have generally decided that paging-in code
works pretty well.  For much internal data, paging in/out works OK,
although there are often funny interactions of paging and garbage
collection.

My bottom line: Mapped Files are Good, but claiming they are almost always
better seems yet to be proved; maybe others will post more data?
	


-john mashey    DISCLAIMER: <generic disclaimer, I speak for me only, etc>
UUCP:    mash@sgi.com 
DDD:    415-933-3090	FAX: 415-967-8496
USPS:   Silicon Graphics 6L-005, 2011 N. Shoreline Blvd, Mountain View, CA 94039-7311