How to Learn Linear Algebra Rigorously for Statistics

[Bayesian1701]

I'm a first-year Ph.D. student (now) and I have discovered that I am behind in my linear algebra skills.  I took the class online over the summer two years ago.  It wasn't taught well at all and was completely applied and covered no theory.  Basically, I know how to find the column space, rank,  determinant, etc.,  but if you ask me to prove anything about those operations I am clueless.  I'm following the proofs in our review material we were given fine,  but I am not at the point where I can make similar proofs independently.  I made the naive assumption that since I have the skills to perform the matrix algebra in a regression that my background was fine, and I am kicking my self in the foot for not learning it this summer. Any advice at learning linear algebra at a rigorous level and while taking linear models at the same time?

 

Edited August 23, 2018 by Bayesian1701

[Stat Assistant Professor]

It's often helpful to study worked out examples and to use WIkipedia, Wolfram, online course notes, etc. to fill in any terminology you may have forgotten or haven't seen before. Most people forget details after they've taken a class and need to refer to online resources or textbooks to help them in their research (apart from WIkipedia, 'standard' linear algebra references include the Matrix Cookbook or "Matrix Algebra from a Statistician's Perspective").

I would look up mathematics basic qualifying exams on advanced linear algebra that have solutions (e.g. UCLA Basic Examination for incoming math PhD students) and try to understand what they did. The more exposure you have to it, the easier it will be to do it yourself. EDIT: Here's a set of qualifying exams that have proof-based linear algebra examples: https://clas.ucdenver.edu/mathematical-and-statistical-sciences/previous-linear-algebra-exams-and-solutions

There are many others posted on other departments' websites.

Edited August 24, 2018 by Applied Math to Stat

[galois]

I doubt you have time to read a separate book during your first year, but I recently purchased some Dover books to brush up on material before entering grad school next year (hopefully ?). I bought Linear Algebra written by George Shilov, translated by Richard Silverman specifically for its axiomatic, proof-heavy feel. It's dense and definitely not a quick read, but as someone who has already seen Linear Algebra before, I think it was a good choice. It's possible that you might be able to dissect it and just read the areas where you need brushing up, but I'm not positive; the layout of the book is definitely different than the curriculum I was taught in undergrad. For example, the first chapter is on determinants.. it seems like an odd choice to me, but I imagine I will appreciate it in time.

[BL250604]

I'm taking a graduate linear algebra class for my senior year to try and learn a little more about the application side. The book we are using is called Matrix Analysis and Applied Linear Algebra by Carl D. Meyer. For only a few dollars extra there is a solution manual which my professor recommended as it helps with some of the more complex problems. 

The author provides some good analogies, and while the book is inherently applied in nature, it does offer some good insight into the theoretical underpinnings. The book assumes a knowledge of linear algebra and multivariable calculus, but does provide some review when introducing new concepts. I personally really like it and find it quite helpful. It is also a quick read, which is nice.

[Statsfan15]

Just wanted to say, you are not alone  I also wish I had reviewed LA this summer. It just hadn't even occurred to me that would be useful.  Everyone I've talked to seems to have had a very different Linear Algebra experience in undergrad... So there are a lot of people feeling the same way right now I'm sure.  I think we will just have to get good at writing proofs quickly... 

[Cal1gula]

This guy on reddit has some good recommendations - https://old.reddit.com/r/statistics/comments/8s9ql7/because_ive_had_to_reference_my_linear_algebra/