2NM4 Week 1: Numerical error and finite precision
Overview
The course introduction frames numerical methods as tools engineers must understand well enough to choose, diagnose, and defend, even when using industrial software, Python, or AI-generated code. The lecture then develops finite-precision arithmetic, roundoff and truncation error, tolerances, and Taylor-based finite differences, showing why smaller steps and apparently converged results can become unreliable.
Key takeaways
- Finite precision is a representation limit, not merely a computer defect: decimal 0.1 has no finite binary expansion, so ordinary floating-point calculations begin with an approximation.
- A small successive-iteration change is not proof of a correct solution; it only indicates that the approximation is no longer changing appreciably, so residuals and tolerances must be interpreted in context.
- Subtractive cancellation makes extremely small finite-difference steps unreliable, despite truncation error decreasing with step size; a useful step balances truncation and roundoff.
- For smooth functions, central differences are second-order accurate while forward differences are first-order accurate, yielding roughly 100-fold versus 10-fold error reduction when the step is reduced by a factor of 10.
- Engineering use of AI-generated code requires the ability to inspect assumptions, recognize numerical failure modes, and defend results, which is why assignments pair open-resource written work with oral technical defenses.
Chapters
- Industrial multiphysics software hides numerical details, which can make failures hard to diagnose without understanding concepts such as scaling.
- A Google Colab prompt generated Python code for an implicit time stepper and Newton solution of a nonlinear heat-transport PDE.
- The demonstration introduces the course challenge: use powerful tools while still understanding and validating their numerical results.
- A prompt-and-response system generates an answer, while an agentic platform can write code, execute it, inspect errors, and revise the code repeatedly.
- Agentic systems can drift from the intended task, so a human must steer and verify their work.
- The lecture treats AI as a widely relevant engineering tool, but emphasizes that generated code is not a substitute for understanding.
- AI is presented alongside the printing press, internet, computers, calculators, and slide rules as a disruptive technology.
- The course encourages learning to use AI while retaining the ability to judge whether its output answers the actual engineering question.
- Engineering often relies on useful approximations, such as spherical-grain models, when added realism is not worth the complexity.
- The course focuses on understanding when to use numerical methods and how to interpret their results, rather than writing every solver from scratch.
- Students will study linear solvers and learn distinctions such as relative tolerance, absolute tolerance, and solution tolerance.
- Large engineering problems are decomposed into smaller computations; nonlinear PDE and implicit time-stepping workflows ultimately rely on linear systems such as Ax = y.
- Students need a computer, Python 3, and the ability to read, write, and execute Jupyter notebooks.
- The first lab introduces Python using McMaster tutorial resources, with teaching assistants available for questions.
- The course provides a free interactive textbook, executable engineering numerical methods in Python, plus prior lecture recordings and transcripts.
- A2L quizzes run every Tuesday and cover the previous week's material; each has four multiple-choice questions and unlimited time and attempts.
- Each quiz is worth 0.5% of the final grade, and the two lowest scores are dropped.
- Unlimited attempts are intended to reward practice rather than penalize students who do not use AI or other shortcuts to obtain an immediate perfect score.
- The course uses a universal-design approach, illustrated by sidewalk ramps that help wheelchair users as well as people pushing carts or carrying loads.
- Five assignments are planned, with one dropped; completing all five is recommended for exam preparation and engineering practice.
- The dropped-score policy reduces the need for some accommodation requests, but students are encouraged to complete work throughout the term.
- Each assignment has a written submission followed the next week by an oral defense with a teaching assistant acting as a technical manager.
- Students may use Python, AI, peers, and other resources for the written work, but must explain their choices without a computer or notes during the defense.
- Written and oral marks are combined geometrically, so a strong report cannot compensate for failing to understand and defend the result.
- The midterm is worth 25%, uses multiple-choice, multi-select, and short-answer questions, and permits one double-sided cheat sheet.
- Students must earn at least 50% on the final exam to pass the course.
- An optional 10-minute research presentation can improve the final-exam component through a 50/50 project-exam average when that average is higher.
- The free textbook includes executable Python examples intended to add practice beyond passively reading worked solutions.
- Running or editing a code cell requires launching a computational backend, or kernel.
- Opening the notebooks in Google Colab allows students to edit code and test how changing inputs affects results.
- Students can propose interactive widgets that illustrate numerical concepts, such as how intersecting lines represent solvable or unsolvable linear systems.
- Technically accurate widget contributions can earn the week's comprehension mark and may be included in the course package with authorship credit.
- Documented, readable GitHub projects can also provide evidence of coding ability for engineering co-op applications.
- A retrieval-augmented Gemini notebook links answers to course lectures and textbook sources, letting students follow citations back to transcripts or recordings.
- The course explores AI-assisted learning while acknowledging students' concerns about dependence and whether they can solve problems without AI.
- AI may help with alternative perspectives, but it should not be mistaken for human counseling or support.
- Writing a number with finitely many digits loses information whenever the exact value has more digits than the representation permits.
- The fraction 1/3 remains exact as a symbolic expression, but writing 0.333... and truncating it means multiplying by 3 no longer necessarily returns exactly 1.
- Symbolic tools such as Mathematica and Maple can preserve unevaluated expressions, but symbolic manipulation can require substantial memory and computation.
- Roundoff error is unavoidable when a value cannot be represented exactly with finite storage, whether the number is rational or irrational.
- Small representation errors can accumulate across repeated operations, making error analysis central to numerical-method design.
- Decimal notation uses powers of 10, while binary notation represents values using powers of 2.
- Binary fractions use negative powers of 2, just as decimal fractions use negative powers of 10.
- The decimal value 0.1 has a repeating, non-terminating binary representation, so standard floating-point storage can only approximate it.
- Common engineering values can therefore carry roundoff error before any substantial computation begins.
- Integers are exactly representable in binary within the range allowed by the available bits.
- Integer storage is still bounded: values outside the permitted range can cause overflow.
- Representing negative as well as positive integers requires a convention for assigning the available bit patterns.
- A common signed-integer convention allocates roughly half of the available bit patterns to negative values and half to nonnegative values.
- For a signed 64-bit integer, the range is from -2^63 through 2^63 - 1, with zero accounting for the asymmetry.
- Overflow occurs when an operation exceeds the representable range; memory-boundary errors can also create serious software security vulnerabilities.
- Data types affect arithmetic: in many languages, dividing two integers can produce integer division, while Python's ordinary division returns a floating-point result.
- NumPy can explicitly define fixed-width integer types, unlike Python's flexible built-in number handling.
- A decimal point in an output is a useful indication that the result is a float rather than an integer.
- Floating-point notation stores a mantissa and an exponent, allowing a wide range of magnitudes with finite precision.
- The mantissa carries the number's sign and significant digits, while the exponent determines the scale and may also be negative.
- In numerical computation, the representation's finite digit count matters more than measurement conventions about trailing significant zeros.
- A normalized binary number has a leading 1 before the binary point, so that bit can be implicit rather than stored.
- IEEE floating-point formats allocate bits for sign, exponent, and mantissa; common sizes are 32-bit single precision, 64-bit double precision, and 128-bit quadruple precision.
- Python's default floating-point values use double precision, while higher precision increases storage and computational costs.
- Converting decimal 0.1 to binary and printing many digits reveals a nearby value rather than an exact 0.1.
- A floating-point number's mantissa limits how many significant digits can be retained, even when its exponent supports a very large or small magnitude.
- Higher precision can reduce roundoff, but should be used when needed rather than assumed to be free.
- Floating-point arithmetic can produce different results when the order of operations changes, even when the expressions are algebraically equivalent.
- Adding a very small value to a much larger value can discard the small contribution before later operations use it.
- Numerical algorithms must account for finite precision rather than relying solely on identities that hold in exact arithmetic.
- Subtracting nearly equal floating-point values can erase their shared significant digits and expose representation noise.
- Machine epsilon measures the smallest increment that changes 1.0; for double precision, it is approximately 2.22 × 10^-16.
- Overly frequent measurements or excessively small numerical step sizes can make differences less reliable because of cancellation.
- The standard quadratic formula can lose accuracy when one root requires subtracting nearly equal large quantities.
- A numerically stable implementation separates cases by sign and uses the relation between the two roots to avoid the cancellation-prone calculation.
- For a finite-difference estimate of the derivative of 2 sin(x) at zero, comparing the result with the known derivative value 2 gives the true absolute error.
- Relative error divides absolute error by the magnitude of the true value, making it more informative when problem scales differ.
- When the true answer is unknown, successive approximations can estimate absolute and relative changes, but small changes only show that the result has stopped improving.
- Absolute and relative tolerances set acceptable error thresholds; a root finder is a special case because its target residual is known to be zero.
- A Taylor series represents a function using derivatives at an expansion point, but computation requires truncating the infinite series.
- Truncation error comes from omitted terms; big O notation summarizes how the leading omitted error scales with a parameter such as step size.
- Taylor series may converge only within a radius, converge slowly, or have large intermediate terms even when they eventually converge.
- The forward-difference derivative uses [f(x + h) - f(x)] / h and has error proportional to h, making it first-order accurate.
- The central-difference formula uses [f(x + h) - f(x - h)] / (2h); Taylor terms cancel so its error is proportional to h squared.
- Reducing h by a factor of 10 should reduce forward-difference error by about 10 and central-difference error by about 100, before roundoff dominates.
- Making the finite-difference step too small increases subtractive cancellation, so accuracy eventually worsens instead of improving.
- A Taylor expansion of sin(x) around zero converges quickly for small x but can have very large intermediate terms for larger inputs.
- For sin(100), reducing the argument modulo 2π before evaluation produces a much more manageable input; standard math libraries use specialized algorithms for this kind of computation.
Transcript excerpt
0:01 Don't need a lobby. Okay. Computational multifysics. We use a software package called com multifysics. Um, convenient name. It generalizes and kind of like hides all the nasty numerical methods from you guys to the point where you can honestly go through and obviously you know 80 of you guys did go through it without understanding actually what's happening under the hood. And that's by design. This is an industrial package, right? and it kind of hides all the nitty-gritty that you don't honestly need to know unless you really need to know it, right? Like at some point it doesn't work and you're scratching your head and your options are either like Google it, ask the the prof like why this isn't working or you got to go back and you actually have to learn like what
0:48 scaling is for example and we didn't even talk about that. So that's the game plan with this guy. Uh so this one is numerical methods for engineering and I'd like to begin with a pop quiz. Okay. So everybody please get out paper and pencil and solve a nonlinear heat transport partial differential equation using an implicit time stepper Newton method. Plot the solution at each time document throughout. Right? You do this. You're done the course. Everybody passes. I'll see you in December. No. Right. Uh, why do people keep popping into the lobby? That's weird. Okay. Obviously, obviously like I'm going out on a on a stick here, right? Um, my background I I've been teaching now,
1:36 this will be my third year teaching at McMaster. I was at Canadian Nuclear Labs working before that. Long story short, CNL is great. There's a lot of cool stuff there, but they were like AI phobic because they have stuff like
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Numerical Methods for Engineering.