Computational Differentiation: Techniques, Applications, and Tools, Proc. AD1996, Santa Fe, New Mexico

SIAM

The book is published by SIAM. More information on it (and an online order form) can be found here.

Preface

This book is a selection of papers from the Second International Workshop on Computational Differentiation held in Sante Fe, New Mexico, February 12-14, 1996, under the sponsorship of the Society for Industrial and Applied Mathematics (SIAM) and the Special Interest Group on Numerical Mathematics of the Association of Computing Machinery. The Sante Fe workshop continued the tradition of the first workshop, also organized by SIAM, held five years earlier in Breckenridge, Colorado. The Breckenridge workshop carried the more conventional term automatic differentiation in its title, a term which is now generally viewed as narrower than the concept of computational differentiation. The latter term encompasses both the automatic transformation of computer programs (automatic differentiation) and methodologies for the efficient exploitation of mathematical underpinnings or program structure.

Research in computational differentiation as represented in this book falls roughly into the three classes of the title:

Techniques	-	the exploitation of the associativity of the chain rule of differential calculus and numerical paradigms to lower the computational complexity of derivative computation.
Applications	-	novel or easier ways to address computational problems enabled by the use of computational differentiation tools and techniques.
Tools	-	the development of high-quality, easy-to-use, general-purpose software systems for the differentiation of computer programs.

The theoretical challenge and practical utility of evaluating functions defined by evaluation programs have only recently become widely recognized. The classical alternatives of computing derivatives in the forward (direct) or reverse (adjoint) mode are now understood to be unidirectional variants of a general scheme for applying the chain rule in a suitable order. This elimination process is performed step by step in floating- point arithmetic, avoiding any kind of expression swell. Nevertheless, the minimization of the total operations count for evaluating gradients, Jacobians, Hessians, or Taylor series turns out to be a difficult combinatorial problem. Hence, one can expect the continued development of heuristic ordering strategies based on program structure and problem-specific characteristics of the underlying computation.

In the application arena, computational differentiation has emerged as a catalyst for the move from simulation to design optimization and data assimilation. The pursuit of these approaches has been made possible by the rapid advance in hardware power. Computational differentiation enables practitioners to quickly generate derivative-enhanced versions of their codes, which are required in these "value-added" uses of their simulation models. The chapters in this volume, which highlight applications in weather modeling, aeronautical engineering, machine tool design, automotive engineering, economics, accelerator design, remote sensing, and electronic device simulation, give a glimpse of the universal utility of computational differentiation, and many more applications can be found in the references.

Both the quality and the availability of software for computational differentiation have improved significantly since the Breckenridge workshop. These achievements are exemplified by the chapters and references in this volume that report on the application of tools to industry-strength problems, as well as by the award of the 1995 Wilkinson Prize to the ADIFOR 2.0 system. Modern general-purpose tools have relieved users from the concerns of mundane questions like language compatibility, and deliver better actual performance and robustness. On the other hand, much remains to be done regarding the treatment of potential nonsmoothness, interactions between commonly used numerical techniques and automatic differentiation, and the implementation of automated trade-offs between storing or recomputing intermediates in the context of multidirectional derivative calculations.

Computational differentiation is an interdisciplinary and international enterprise. Over the past few years, some common terminology has evolved, and some artificial barriers (for example, vis-á-vis the computer algebra community) have been broken down.

The enlarged scope of the field of computational differentation, as in the fields of solving linear equations and integrating initial value problems, means that textbook methods, while theoretically still applicable, have been rendered ineffective and must be superseded by more advanced techniques. The analysis and implementation of such computational differentiation techniques provide challenges for hard research and development well into the next millennium.

The Second International Workshop on Computational Differentiation owes its success primarily to the farsightedness of the SIAM board and the personal efforts of Gail Corbett, Trini Flores, Lillian Hunt, Vickie Kearn, Bill Kolata, Maryann Donaghy, Diana Peirce, and Colleen Robishaw at SIAM. We are also grateful for the support of John Gilbert and Julie Goetz from the cosponsoring Special Interest Group on Numerical Mathematics (SIGNUM) of the Association of Computing Machinery. We acknowledge the financial support provided by Kamal Abdali through the Numeric and Symbolic Computation Program at the National Science Foundation,- Fred Howes through the Mathematical, Information, and Computational Science Division subprogram of the Office of Computational and Technology Research; the U.S. Department of Energy; Rick Stevens through the Mathematics and Computer Science Division at Argonne National Laboratory,, Michigan State University, and the Alfred P. Sloan Foundation. The excellent computing and communications facilities at the workshop were facilitated by Ann Hayes of the Advanced Computing Laboratory of Los Alamos National Laboratory, with support from Sun Microsystems and the Numerical Algorithms Group.

We could not have succeeded in preparing this volume without the considerable editorial skills of Gail Pieper and the continuous support of program committee members Alan Carle, Bruce Christianson, Louis Rail, and Nicole Rostaing-Schmidt. We also gratefully acknowledge the support of members Larry Green, Jorge Moré, and Detlef Laugwitz in selecting talks and posters.

Table of Contents

1

An Introduction to Automatic Differentiation
Louis B.Rall and George F. Corliss

19

Calculus and Numerics on Levi-Civita Fields
Martin Berz

37

Exception Handling in Derivative Computation with Nonarchimedean Calculus
Khodr Shamseddine and Martin Berz

57

Jet Space as the Geometric Arena of Automatic Differentiation
Gordon D. Pusch

63

Remainder Differential Algebras and Their Applications
Kyoko Makino and Martin Berz

75

Automatic Differentiation of Conditional Branches in an Operator Overloading Context
R. Baker Kearfott

83

Hierarchical Approaches to Automatic Differentiation
Christian H. Bischof and Mohammed R. Haghighat

95

Optimal Time and Minimum Space-Time Product for Reversing a Certain Class of Programm
José Grimm, Loïc Pottier, and Nicole Rostaing-Schmidt

107

Sharing Storage Using Dirty Vectors
Bruce Christianson, Laurence Dixon, and Steven Brown

117

Splitting of Algebraic Expressions for Automatic Differentiation
Christéle Faure

129

A Symbolic Code Generator for Parameter Estimation
Guy Chavent, Jérôme Jaffré, Sophie Jégou, and Jun Liu

137

Parallelism in the Reverse Mode
Jochen Benary

149

Structure and Efficient Jacobian Calculation
Thomas F. Coleman and Arun Verma

161

Automatic Computation of Sparse Jacobians by Applying the Methods of Mewsam and Ramsdell
Uwe Geitner, Jean Utke, Andreas Griewank

173

More AD of Nonlinear AMPL Models: Computing Hessian Information and Exploiting Partial Separability
Daivd M. Gay

185

Automatic Adjoint Modeling within a Program Generation Framework: A Case Study for a Weather Forecasting Grid-Point Model
Victor V. Goldman and Gerard Cats

195

Accounting for Model Error in Data Assimilation Using Adjoint Models
Anne K. Griffith and Nancy K. Nichols

205

Automatic Differentiation as a Tool for Sensitivity Analysis of a Convective Storm in a 3-D Cloud Model
Seon Ki Park, Kelvin K. Droegemeier, and Christian H. Bischof

215

Automatic Differentiation and Implicit Differential Equations
Stephen L. Campbell and Richard Hollenbeck

229

Computer Proofs for Bifurcations of Planar Dynamical Systems
John Guckenheimer

239

A Differentiation-Based Approach to Dynamic Simulation and Optimazation with High-Index Differential-Algebraic Equations
William F. Feehery and Paul I. Barton

253

Efficient Newton Steps without Jacobians
Jean Utke

265

Estimation of Econometric Functions
Max E. Jerrell

273

Generalized Neural Networks, Computational Differentiation, and Evolution
David W. Juedes and Karthik Balakrishnan

287

Automatic Differentiation Apllied to Convex Optimazation
Eric Hassold and André Galligo

297

Comparisons of Different Automatic Differentiation Tools in Circuit Simulation
Wolfram Klein

309

Automatic Differentiation in Direct and Reverse Modes: Application to Optimum Shapes Design in Fluid Mechanics
Bijan Mohammadi, Jean-Michel Malé, and Nicole Rostaing-Schmidt

319

Adjoint Variable Method for Sensitivity Analysis of Multibody Systems Interpreted as a Continuous, Hybrid Form of Automatic Differentiation
Peter Eberhard

329

A new Method for Bevel Gear Tooth Flank Computation
Ulf Hutschenreiter

343

Improving Derivative Performace for CFD by Using Simplified Recurrences
Alan Carle and Mike Fagan

353

An Implementation of the Forward and Reverse Modes of Automatic Differentiation in Maple
Michael B. Monagan and René R. Rodini

363

COSY INFINITY and Its Applications in Nonlinear Dynamics
Martin Berz, Kyoko Makino, Khodr Shamseddine, Georg H. Hofstätter, and Weishi Wan

367

PADRE2 - Fortran Precompiler for Automatic Differentiation and Estimates of Rounding Errors
Koichi Kubota

375

ADOL-F: Automatic Differentiation of Fortran Codes
Dimitri Shiriaev and Andreas Griewank

385

User's Experience with ADIFOR 2.0
Christian Bischof and Alan Carle

393

Bibliography of Computational Differentiation
Wenhong Yang and George F. Corliss

419

Index

M. Berz, C. Bischof, A. Griewank, G. Corliss (Eds.), SIAM, 1996, ISBN 0-89871-385-4

Go Back to the reprint server.
Go Back to the home page.

This page is maintained by Kyoko Makino. Please contact her if there are any problems with it.