The Cybernetic Perspective

In the original sense, "cybernetics" means optimal use of information in decisionmaking.

It emerges that to develop a statistics of information requires a change of perspective from the simple one where developing the model is informed by the system under study to one where the model and the system under study are two components of a larger system. The Wiener filter and its descendant the Kalman filter are the best known examples of this approach but it is more general.

I believe this use of the word "model" is not entirely coincident with the software engineering sense as in "model-driven development" though it does mesh more or less well with "Model-View-Controller". In our world the View is relatively trivial and usually done offline as postprocessing; it's the utter absence of the Controller that troubles me.

(Update: That last is really a stretch on second thought. Probably more sellable is the idea that the M in MVC corresponds to system state (variables) and the C corresponds to executable code (statements). In the end the word "model" is just as much a source of confusion as ever.)

Anyway I think the cybernetic perspective has value in getting climate modeling unstuck.

Quote of the Day

Chris McAvoy on the ChiPy list:

I don't know for sure, but from a "Zen of Python" perspective, when I
can't do something with Python it usually ends up that what I'm trying
to do shouldn't be done.

Life in the Climate Trenches: Part 1

I've got not one but two build problems going on. If the rest of my life weren't going well I'd be awfully depressed. I really hate wasting time on this sort of mundane frustration.

Lonestar (the supercomputer) is offline for alternate Tuesday maintenance, so for today I will share build Probleme Numero Un with you. I welcome any skill transfer that can help untangle all this; comments welcome. Rather than polluting the blogspace with innumerable followups I will just provide updates to this entry.

Tomorrow I'll start documenting Probleme Numero Deux: running a standard CAM use case.

It may be that we'll more be documenting my own ignorance than any fundamental problem with the software from some point of view; that will be a learning experience just the same.

You may object that this isn't the sort of thing you want your tax dollars paying PhD's to fumble around with; in that case I would be in agreement with you. In the private sector this would clearly be a systems task and not a scientist task. So I appeal to your mercy as a lousy admin and a potentially productive scientist, albeit of a somewhat eccentirc stripe. The sooner I get these things done the sooner I can be a scientist.

OK, here's the scoop (unix/posix skills required to make any sense out of this):

To participate in NCAR CAM development we need to use their diagnostic package which in turn depends on NCL. Having a legitimate connection to a US research institution, I have long since jumped through the hoops and am registered on the Earth System Grid. So I download the package and all its dependencies.

OK, so the UTIG disk systems are all cross-mounted and so, though I have root on the climate research machine I don't have write privileges on /usr . Our excellent systems folk have set it up so that climate users just define $CLIMATE in their .cshrc which will invoke anything I put (or conceivably somebody else puts) into /usr/local/climate so all I have to do that is nonstandard is insert "climate/" into the path; I make my own ../bin ../lib ../man in there; everyone interested in climate ends up with it in their path, nobody else has their namespace messed up, a very nice approach.

Alas, the lengthy but very clear build instructions do not work (this is standard practice with NCAR products alas; apparently they lack the skills and/or funding to actually make their products portable, though the build instructions are usually very hopeful.) In the present case the snag occurs even before I get to NCAR's stuff.

NetCDF is already available. I have acquired and build jpeglib and zlib as follows:

cd jpeg-6b
make clean
./configure --prefix=/usr/local/climate
make
make install

which puts five executables into /usr/local/climate/bin and

cd zlib-1.2.3
make clean
./configure --prefix=/usr/local/climate
make
make install

which puts a library into /usr/local/climate/lib

My current snag is building HDF-4, which in the configure phase always stops at:

checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for compress2 in -lz... yes
checking jpeglib.h usability... no
checking jpeglib.h presence... no
checking for jpeglib.h... no
checking for jpeg_start_decompress in -ljpeg... no
configure: error: couldn't find jpeg library

I have tried various combinations here. What seems to me ought to work is

./configure --prefix=/usr/local/climate --with-zlib=/usr/local/climate \
--with-jpeg=/usr/local/climate --disable-netcdf

but it seems to ignore the prefix altogether. I change --with-zlib to point to a totally bogus address, and yet it finds the zlib stuff anyway. It seems to be ignoring the flags I am setting.

Working hypothesis: I am building jpeg wrong, and some system setting is overriding my value of --with-zlib.

A less satisfying but easier solution could come from downloading binaries. At this point I simply have to swallow my pride and ask which if any of the binaries listed here might work.

Advice would be even more welcome than sympathy.

Update 1/24:

UTIG support advised option 4, which worked more or less smoothly.

Also, from UTIG support:

Oh I didn't see you had your own (up a directory). The command you
were looking for would be

./configure --prefix=/usr/local/climate \
--with-zlib=/usr/local/climate/NCLc3/zlib-1.2.3 \
--with-jpeg=/usr/local/climate/NCLc3/jpeg-6b --disable-netcdf

And I'm idiotically proud of this script, my first conditional in a shell script in a long time:

set hn = `hostname`
if ($hn == 'foo.whatever.utexas.edu') then
echo "climate paths added"
setenv NCARG_ROOT /usr/local/climate/NCLBin/
set path = ( /usr/local/climate/NCLBin/bin/ $path )
endif


So there's finally a happy ending on this one. TACC is still having hardware or configuration problems with Lonestar which is preventing any progress on the other front.

Language of Choice

Presumably everyone knows that Python is on the rise with a bullet. Of course, now that Python is sexy it will probably attract some weaker programmers, diluting its reputation. Such are the costs of great beauty. Right now, though, the upswing seems to be near its maximum rate.

In the Slashdot article discussing this I saw a ref to a lexicon definition the concept of Language of Choice in the Jargon File. FORTRAN is dismissed as a niche language for scientists, but nothing better is proposed.

So what should the language of choice be for high performance numerical computing?

There are teams working on replacements (Fortress, Chapel, some IBM thing with a forgettable name), but I think much of the high performance work of the future can be done in Python with a code generation strategy. I'm not sure what the target language ought to be for the code generation. Would there be advantages to Fortress or Chapel for this? For contemporary purposes even F77 would work (as I said in the PyNSol papers) but to be honest the F77 target is almost a stunt, just to make a point. I don't expect it would be useful in the long run.

I am currently thinking most of the system should be agnostic as to the target language. If done right, multiple targets can be supported in an architecture-dependent bottom layer. Even Verilog is a candidate. ( F90 and onward are not. I'd sooner use BF or Whitespace. )

Update: Bryan Lawrence has a nice summary of the issues.

Do NCAR or GISS Ask for This?

This crossed my path on a CS mailing list I follow. The context is about whether students need more software engineering or more computer science. It's a bit harsh on us older types but it certainly is an interesting list. I post it so scientific computing types can consider what they may or may not be missing. I should also note that there are practicing scientific coders for whom I have great respect who would find this list ludicrous.

As a software engineer in the industry and as a person who hires developers I have to say that 2 out of every 3 interviews I take from people with a CS or MS in computer science I do not hire because they are completely inadequate as software engineers. More often than not, they do not know about or have distaste for one or more of the following:

• Design Patterns
• Agile engineering / programming
• Extreme Programming
• Test Driven Development
• Composition vs. Inheritance

We also give a programming test as a part of the interview process, it is mathematically / algorithmically moderate in challenge and most interviewees (80%) can solve the problem, but only 15-20% solve it with good programming practices and maybe 5-10% write unit tests. Actually, if an interviewee who has good engineering, but does not solve the problem is more appealing than someone who solves the problem.

I do recall from my undergraduate program at Purdue that we did have a "Software Engineering" class that did try to drive home the points I mentioned, but I have to say that it did not go far enough.

I would never diminish CS courses, mathematics, or general studies programs ever. I would say that anyone who only takes their CS and never masters the engineering principles either through other course work (several hours worth) or through an internship does not stand much of a chance getting hired at the company I work for.

In the end, I would say that if I knew 6 years ago what I knew now, I would have gone for the software engineering program and tried for a dual major in CS if I had the bandwidth.

For what it's worth I believe that design patterns in practice are overstressed, and that the list isn't especially elegant or complete. That said I also have the impression that these ideas are only very dimly perceived in the climate modeling centers and a great deal of benefit could be gained from them.

Progress

OLPC is a great achievement if only for this story:

"Microsoft struggles to port Windows to a device originally conceived to run Linux."

Still waiting on mine...

Why is Climate Modeling Stuck?

I believe that progress in climate modeling has been relatively limited since the first successes in linking atmosphere and ocean models without flux corrections. (That's about a decade now, long enough to start being cause for concern.) One idea is that tighter codesign of components such as atmosphere and ocean models in the first place would help, and there's something to be said for that, but I don't think that's the core issue.

I suggest that there is a deeper issue based on workflow presumptions. The relationship between the computer science community and the application science community is key. I suggest that the relationship is ill-understood and consequently the field is underproductive.

The relationship between the software development practitioners and the domain scientists is misconstrued by both sides, and both are limited by past experience. Success in such fields as weather modeling and tide prediction provide a context which inappropriately dominates thinking, planning and execution.

Operational codes are the wrong model because scientists do not modify operational codes. Commercial codes are also the wrong model because bankers, CFOs and COOs do not modify operational codes. The primary purpose of scientific codes as opposed to operational codes is to enable science, that is, free experimentation and testing of hypotheses.

Modifiability by non-expert programmers should be and sadly is not treated as a crucial design constraint. The application scientist is an expert on physics, perhaps on certain branches of mathematics such as statistics and dynamics, but is typically a journeyman programmer. In general the scientist does not find the abstractions of computer science intrinsically interesting and considers the program to be an expensive and balky laboratory tool.

Being presented with codes that are not designed for modification greatly limits scientific productivity. Some scientists have enormous energy for the task (or the assistance of relatively unambitious and unmarketable Fortran-ready assistants) and take on the task with energy and panache, but the sad fact is that they have little idea of what to do or how to do it. This is hardly their fault; they are modifying opaque and unwelcoming bodies of code. Under the daunting circumstances these modifications have the flavor of "one-offs", scripts intended to perform a single calculation, and treated as done more or less when the result "looks reasonable". The key abstractions of computer science and even its key goals are ignored, just as if you were writing a five-liner to, say, flatten a directory tree with some systematic renaming. "Hmm, looks right. OK, next issue."

This, while scientific coding has much to learn from the commercial sector, the key use case is rather atypical. The key is in providing an abstraction layer useful to the journeyman programmer, while providing all the verification, validation, replicability, version control and process management the user needs, whether the user knows it or not. As these services become discovered and understood, the value of these abstractions will be revealed, and the power of the entire enterprise will resume its forward progress.

It's my opinion that Python provides not only a platform for this strategy but also an example of it. When a novice Python programmer invokes "a = b + c", a surprisingly large number of things potentially happen. An arithmetic addition is commonly but not inevitably among the consequences and the intentions. The additional machinery is not in the way of the young novice counting apples but is available to the programmer extending the addition operator to support user defined classes.

Consider why Matlab is so widely preferred over the much more elegant and powerful Mathematica platform by application scientists. This is because the scientists are not interested in abstractions in their own right; they are interested in the systems they study. Software is seen as a tool to investigate the systems and not as a topic of intrinsic interest. Matlab is (arguably incorrectly) perceived as better than Mathematica because it exposes only abstractions that map naturally onto the application scientist's worldview.

Alas, the application scientist's worldview is largely (see Marshall McLuhan) formed by the tools with which the scientist is most familiar. The key to progress is the Pythonic way, which is to provide great abstraction power without having it get in the way. Scientists learn mathematical abstractions as necessary to understand the phenomena of interest. Computer science properly construed is a branch of mathematics (and not a branch of trade-school mechanics thankyouverymuch) and scientists will take to the more relevant of its abstractions as they become available and their utility becomes clear.

Maybe starting from a blank slate we can get moving again toward a system that can actually make useful regional climate prognoses. It is time we took on the serious task of determining the extent to which such a thing is possible. I also think the strategies I have hinted at here have broad applicability in other sciences.

I am trying to work through enough details of how to extend this Python mojo to scientific programming to make a credible proposal. I think I have enough to work with, but I'll have to treat the details as a trade secret for now. Meanwhile I would welcome comments.