Okay, thank you Jan for your participation in this project on the history of psychometrics. I'll ask questions about three themes, your career, the relation between psychology and psychometrics or other fields than psychometrics, and of course your view on the history and future of psychometrics. And I always start with the question, how did you end up in psychometrics?
I started in psychology in Leiden in 1963, and I got fed up with it in 1964 maybe, pretty quickly, so I switched to physics for a year, but I didn't feel like doing experiments. So then I became interested in statistics and mathematics, and I was hired by Len de Klerk in experimental psychology. It was not really at the same time, that was in 1965 or 1966, and he was working with John van de Geer. John van de Geer was setting up the Department of Data Theory at Leiden University, he was dean of the social sciences at the time, and was setting up his own department. I was the first one he hired into this new Department of Data Theory; first as a student assistant, and later as an assistant professor. Of course, the Department of Data Theory did psychometrics. There was some mathematical psychology going on initially, but pretty soon all we did was psychometrics.
But data theory can also imply all sorts of data, right? Was there a reason why it was mainly psychometric?
Well, van de Geer of course was interested in multivariate analysis and on a level pretty far removed from actual applications, although there were always actual applications around. But the emphasis was always on developing multivariate analysis techniques, and that was fine with me, so I went along with that.
But you started off with psychology..
As a major, yes.
As a major. But you prefer psychometrics?
I was not interested in psychology anymore, after a year. I didn't like the idea that everything was, well, let's say debatable, or uncertain, or up in the air, or whatever the appropriate term is.
You preferred real knowledge.
Well yes, 'real knowledge', though not empirical knowledge, but mathematics basically.
You mentioned there were sometimes applications, in this data theory department, or people worked on applications; did you also still work on psychological research?
There were psychologists who sometimes needed something, like scaling techniques or data analysis efforts, and they were basically clients of the Department of Data Theory, and they were helped with whatever they needed, either something that already existed or something new. But there wasn't that much consulting, it was a somewhat isolated and luxurious department within the social sciences.
What about it was so luxurious?
That we were not very often bothered by things which had to be done, or something like that. There wasn't any real teaching, there was some consulting but only minimal, so basically we could spend all our hours on developing techniques.
And you did your Ph.D. there as well right?
Yes, I did my Ph.D. in 1973. I was invited to go to Bell Laboratories in New Jersey, and it was unwise to go to the US without having a Ph.D., because then you could not get an academic appointment basically. So I finished that just before I went. The dissertation was called Canonical Analysis of Categorical Data. It's more or less clear from the title what it's on.
Well, can you still explain it for people like me, who don't fully understand the title?
It was the precursor of what was later known as the Gifi System, which is a large reorganization of descriptive multivariate analysis techniques in such a way that they apply both to numerical and non-numerical data. That covers regression and principal components analysis, factor analysis, all those techniques in one common framework and ultimately in a series of computer programs as well. My dissertation was basically the first programmatic statement of that program. The Gifi project itself ran until about 1990, I left in 1987, but there was still some ends to wrap up and the Data Theory group in Leiden that still existed at the time, wrapped up those loose ends. They produced a book, the final version of the Gifi book
That was one of your major topics right, in your career, Gifi.
Certainly in the first part of my career, after I came back from Bell Laboratories basically, so that was in 1974. We started developing the computer programs in 1974, at the time we were housed in the computing center of the University, on the Wassenaarseweg, and that grew into the Gifi project. We got pretty large grants from NWO, the basic Netherlands grant organization for sciences, and we started hiring people. Ultimately the Gifi group was about maybe 10 - 15 people who were working on the postdoctoral course that we taught in 1980 and 1981, and on the book that eventually came out in 1990.
The book was named after a certain person right.
It was named after a butler, Francis Galton's butler.
Why Francis Galton's butler?
The reasoning was that Francis Galton's butler got a raw deal from Galton, when Galton died. He served him for 25 years, he only got a couple of hundred pounds, and most of Galton's fortune, which was considerable, went to establishing the first chair of statistics, he donated a chair to University College. He only gave his faithful Swiss butler 500 bucks. So we needed a pseudonym, and we chose Gifi. It's similar to 'Meerling', which were two books that were also written by a collective of authors, mostly in the methodology department of psychology, but data theory was also involved, and they also adopted a pseudonym, Meerling. Now, Meerling was not a real person obviously, it just means 'multiple persons' in Dutch.
So you wanted to honor this butler?
Yes. We also made it some sort of a little bit of a riddle, because we didn't publish initially who he actually was. There was a picture of him in the book but people still didn't know who he was. It was an old picture of a Victorian type of person; that was more mysterious than revealing who it actually was.
And is your Gifi work still finding applications?
Yes, as far as I'm concerned, the project is still going on. We have a group which now consists of Patrick Mair, who is a professor at Harvard, and Patrick Groenen who is a professor at Erasmus, and me, and we're sort of doing a reorganization, a reworking of Gifi with new computer programs, new theory to some extent, and new algorithms. So that's still on going. It's not a group of 25 people anymore, but it's one of the things that I'm doing in retirement.
So you're not really retired. You are officially, but you're still working.
I'm not getting paid anymore, or something like that, I have my retirement money, but I don't have to do any other things anymore, let me put it that way. I don't have to teach anymore, I don't have to go to meetings.
So you do this for your own, because you enjoy it, basically.
Yes!
When you went to the Bell Labs, after your PhD, what work did you do there?
Same type of work. One of the things that was going on at the time, was the work in, what's it called, nonmetric multidmensional scaling, which was started by Shepard and Kruskall who were at Bell Labs. Shepard was there in 1962, Kruskall in 1964. That received a lot of attention at the time, and van de Geer and his students, of which I was one, were developing various versions of these nonmetric multidimensional scaling programs. Eddie Roskam was another student of van de Geer who worked in the same area. I started writing up this work when data theory was formed in 1968, I put out enormous numbers of internal reports with nice red covers, and they caught the attention of the people at Bell Labs, because they were still working on multidimensional scaling. So they invited me over to work on multidimensional scaling, nonmetric scaling, various multivariate analysis techniques, and I spent a year there, in New Jersey, with my family.
Did you become assistant professor after?
I was already assistant professor. I'm not entirely sure about the time scale, but I think I probably became, what was then called lector, in maybe 1975 and at some point, lectors were converted to full professors, in maybe 1976, 1977 something like that. So I became professor, and then eventually, Van de Geer stepped back and I became the chair of data theory, which was around 1980. We also started to organize these Gifi post doctoral courses, and building out that project, until I left in 1987.
And then you went Los Angeles.
I was invited by UCLA to build a statistics graduate program in the Division of Social Sciences. There was some sort of initiative by the Dean of Social Sciences to build out the quantitative components of the various social science departments, so they needed somebody in charge of that, of that effort. It was an open application, but eventually they chose me, and I got an appointment partially in psychology and partially in mathematics, so I was professor of mathematics and of psychology, starting in 1987. And then when I arrived there, it turned out that most of the faculty didn't really want a division or a graduate program in statistics for the social sciences, but they wanted a Department of Statistics, because there wasn't one at the UCLA. So then we surreptitiously, maybe against the wishes of the Dean of Social Sciences, started our effort to build a statistics department at UCLA. That took about 11 years, but eventually in 1998 we got a Department of Statistics and I became the chair of the department of statistics until my retirement, which was three years ago.
Did the statistics department also encourage psychometric research?
I was president of the Psychometric Society in I think 1987, which is also when we had the Psychometric Society meeting at UCLA, but probably before that, data theory was already quite far removed from psychology. If anything, it was in the social sciences, organizationally it was in the social sciences as well, we had people working there from various social sciences and from mathematics and from various other disciplines as well. Although it was nice to organize the Psychometric Society meeting, I had moved away from psychometrics proper, quite some time before I became president of the Psychometric Society. The type of multivariate analysis work that I was doing at the time and still am doing, typically originates with psychometricians. It's the same type of work that Henk Kiers, Jos ten Berge, Patrick Groenen, and Willem Heiser are doing. So, in that sense I haven't moved away from psychometrics that dramatically, but organizationally, all my efforts were to get a statistics department established, and as a consequence, my peer group and the people I had to talk to and the people I had to influence, were statisticians. So consequently, I moved most of my efforts to statistics and I started websites about statistics, electronic textbooks for statistics, journals that were statistics journals, all to support this effort to get a department of statistics at UCLA.
Did your research deal with other types of data than psychometric data?
Sure, yes. Medical data, astronomy data, satellites, traffic data, bank data, credit cards, anything, name it. Because there was now a general statistics department for the whole campus and we had a consulting center established early on, we got clients from all over the university. Probably most clients came from the medical sciences actually, but they also came from astronomy, from chemistry, from all kinds of departments.
I reckon Peter Bentler is also part of the department?
Peter Bentler has a joint appointment in Psychology and Statistics, so he's half psychology, half statistics.
The statistics department at UCLA is not more focused on psychometrics than on statistics in general.
The statistics department at UCLA is not in the social sciences, or in the behavioral sciences. Psychology is in the behavioral sciences, or the life sciences, and the social sciences are their own division in the college. Statistics is in the division of physical sciences, organizationally we're more related to astronomy, chemistry, physics, and mathematics, but clients come from all over the place. Of course, it has shifted over the years, everything has become much more computational and much more data oriented than it used to be.
Do you think that's a good development?
Oh yes, I think that's a wonderful development! It's basically, in a sense, a dream come true. I was hired over other people for that particular position because they knew that I emphasized computation and multivariate analysis and data analysis, and the computational side of statistics. And at the time, there was sort of a war going on between mathematical statistics and computational statistics.
For the layman here, what exactly is the difference between the two?
Mathematical statistics is basically a branch of applied mathematics, where your business is to prove theorems, and computational statistics is a branch of computation where your product is a technique, a computer program, and to analyze data obviously. That battle raged for quite a long time. It's over now, I mean, the computational people have won, which is a good development, and especially good for UCLA statistics because we had that emphasis from the start, even before it was popular, or before that war was over.
When people ask you what your profession is..
Statistician.
Without a doubt.
It's also my title obviously, a Professor of Statistics.
Some psychometricians affiliate strongly with psychology, or find it interesting, but you..
I have basically not kept up or looked at any psychology journals since 1968 maybe, , since I stopped taking psychology lectures and things like that. So there's no special tie with psychology. Obviously, if you're a psychometrician, you get into contact with test theory and IRT, but I'm not sure if that's psychology or not.
I think people differ on that.
I would imagine, yes.
At least, it's about psychological data.
Yes, or educational data.
Or educational data, fair enough.
For a couple of years I was the editor of the Journal of Educational Statistics as well. That was a contact I had with the education people, I think that was around 1991, 1992, and I went to some education oriented events, but I was still in the company of statisticians, mostly. I'm still working with Patrick Mair, who's a professor of statistics in psychology or in the behavioral sciences, and with Patrick Groenen who is an econometrician, although he has basically had psychometric training, he is a data theory product, so there are still those connections. But I haven't been to a Psychometric Society meeting since the one that I organized in 1987.
Do you still read Psychometrika?
I've had a subscription for a couple of times, but I've renewed generally when I realized that I had let it lapse. I don't think I'm subscribed at the moment as well. I usually look at the table of content, and if there's something written by one of my old friends I probably look at it a little bit more, but it's not what I normally read. If you look at the bookcase, it's all mathematics, computation, not even statistics actually; it's all matrices, optimization, and mathematics. There's no psychometrics in there.
Is mathematics your true passion?
I think my true passion is programming, or computation. I'm interested in pure mathematics in the same sense as I'm interested in art: not as a profession, but as somebody who observes it and enjoys it, like poetry or something like that. The actual work is in the programming, in the computation.
Should psychology become more statistics oriented?
Especially these days that's a very loaded question. It's always been a loaded question, but now, there's this replication crisis of course, and the replication crisis has as its main message that the statistical methods used in psychology are bad, bogus, not very conducive to building up a cumulative science. It's something that I have written on in the past, quite a long time ago, but it's now coming more to the forefront because of the ESP experiments and things like that, so I think, as a sort of general observation, as somebody who is not really involved in the debate, it's probably necessary that psychology thinks very hard about the statistical methods it uses at the moment, and changes something in their paradigms. I'm not entirely sure what that should be because again, I don't really follow the debate except at the level of Science or Nature, or those more general journals that write on this crisis.
But regardless of the reproducibility problem, you think that statistics might be a good framework for psychological research?
Basically, any discipline that collects data will benefit from what used to be called statistical techniques and is now often called data analysis techniques or data science techniques. There are all these new names that are mostly bogus because they're just there to make money, to get grants and things like that, but the idea is still the same. You have data, you want to present them in such a way that they convince people of something that is either true or not true, and you want to do that in a way which conveys as much information as possible, and is still readable and convincing. So if that is statistics, and I would like to think that it is, then every science that collects data, so not necessarily philosophy, will benefit from these techniques.
Is it the job of psychometrics to improve the statistics in psychology?
I would think so, yes. Ever since the standard paradigm for experimental psychology was established, so analysis of variance, t-tests, there has been pretty heavy criticism on it, such as the critique on significance levels, which generally has been ignored. It's now coming back to bite them in the posterior. Psychometrics is not different from chemometrics and econometrics and other metrics, many disciplines have a similar type of activity as psychometrics is in education and psychology, but I think generally their job is to teach economists or psychologists or chemists how to handle data, how to analyze data, and obviously, how to do it well, because that's the whole idea of teaching something. So yes, if psychology needs better statistical techniques and all the indications are that they do, then psychometricians should play a role in teaching them how to do that.
Should psychometrics also play a role in actually building psychological theory, or should that be up to the psychologist?
I think that should be up to the psychologist. There's something like mathematical psychology, which is, if I understand it correctly, not in a very good state but it used to be seen as the summit of scientific psychology. So mathematical psychology is totally psychology, and doesn't have anything to do with psychometrics, accept for the fact that they both use mathematics, but so does physics; using mathematics doesn't say anything in itself. So, no, I don't think psychometricians should be involved in developing theories. Of course, a particular person can be both a psychologist and a psychometrician, there's no law against that. Someone could spend half of his time developing psychological theory and half of his time developing statistical techniques to analyze psychological data.
But as its main purpose, psychometrics doesn't have to be involved in theory building.
No, it doesn't, the definition doesn't involve developing psychological theory.
So the 'psycho'-part in psychometricians just relates to the type of data that psychometricians deal with?
No, it refers to the type of clients psychometricians have, which is often the same thing as the type of data they use, obviously. I mean, tests are the obvious type of data that they have, for example attitude scales, but I guess also data that come from experimental designs.
Is that true? I think test data is probably what they mostly use.
Yes, there has always been a tension between the type of statistics used by experimental psychologists and the type of statistics that psychometrics is producing, and that's reflected in journal acceptance policies, in a lot of polemics over the years. Eventually it's all about analyzing data, and in the right sense, it also involves collecting data and setting up experiments and those kinds of things, but everything that has to do with data should be within the scope of psychometrics. There's very little about this experimental approach to psychology in Psychometrika for instance, but that's just for historical reasons I guess.
Psychometrics, should, could, spread its wings a little bit more.
Yes, I think so, especially to alleviate the current crisis maybe. But I won't be involved in that.
Do you think psychometrics could play a role in other fields as well?
Psychometrics also has a very problematic relationship with statistics. The first generation of big names in psychometrics mostly concentrated on factor analysis, and factor analysis did not have a good name in statistics for a very long time, for obvious reasons. It's a somewhat strange technique, and people wrote about it as if it was sent directly from God. There were all kinds of strange practices and weird computations, so, people who should've helped making it better, instead decided to sit on the sidelines and criticize it, so that didn't really help. And then there are a lot of things published in psychometrics that have been done better elsewhere, and there are a lot of things published in psychometrics that actually is original and up to date and innovative and ahead of its time, in the sense that other techniques or other sciences developed it later. But then generally if the other sciences developed it later, they ignored things that were going on in psychometrics 10 to 15 years earlier. So, it's an insulated field. That's a bit of a problem, it's insulated from psychology to a large extent because psychology is dominated by the experimental tradition and not by psychometrics, and it's insulated from statistics because it has not really tried to penetrate the much larger field that is statistics. If psychometricians had tried to publish more in regular statistical journals then they would have prevented some of these isolation problems, and it would've been possible I think as well. It would've been hard because there's always this protective shield around each discipline, but it would have been possible. And to some extent, it's happening more and more: there's more overlap between fields now than there used to be.
Do you have an example of something that was published first in psychometrics and then was picked up years later somewhere else?
There are many examples. Factor analysis is maybe the first example. It was often presented in a non-mathematical way, a non-rigorous way, but eventually it found its way into statistics and was transformed in the process into something more respectable and more accepted. But there are many other examples. Non metric multidimensional scaling or multidimensional scaling in general started in psychometrics, test theory started in psychometrics, three-way data analysis started in psychometrics, and many of the results of that are still continually being rediscovered in chemometrics.
I'm not a statistician, so can you explain what makes factor analysis such a strange analysis for a statistician?
It was mostly the way the original factor analysts, who were psychologists, like Spearman and Cattell, presented it as some magical tool that could discover laws of nature by simple inductive data analysis. The other thing was that the actual procedures, the actual computations that were done in factor analysis were from a statistical point of view fairly primitive. That's exactly where statisticians could've jumped in and improved the procedures, but they didn't do that until the 1940s and the 1950s, and then it was still an isolated example. It didn't really happen until Jöreskog which was in the 1970s. So it was partly the claims that people made, partly the fact that from the philosophy of science point of view - the idea that you discover these deep underlying constructs just by computation - and partly, the actual quality of the work from a mathematical point of view, that prevented the technique from being accepted. Part of it was probably also because they concentrated on intelligence, and that has always been a problematic concept.
You have not worked on intelligence?
I have some publications about the IQ debate as it was called, together with Jos Jaspers, a professor of social psychology in Leiden at the time. Not much, but there's also some other stuff in psychometrical genetics. At the time I was pretty interested in what was then known as the IQ debate, I'm not entirely sure if that's still the name of it, but I published some papers on that topic, and then of course, when I was editing the Journal of Educational Statistics, I had to be interested in test theory and similar things. So I published quite a bit on education as well, but always with a slant of data analysis and multivariate analysis.
What do you consider your biggest achievement? What you'll be remembered for?
I don't know how long I'll be remembered!
Who knows . What are you most proud of? Perhaps that's a better question.
The UCLA Department of Statistics is probably the most permanent thing that I helped establish and I was sort of instrumental in having it established. It's interesting, because it started small obviously, it started slowly because there was a lot of opposition, and then you see it growing, and now there are about 300 to 400 people involved, students and faculty and staff, so I think that's large accomplishment. I obviously didn't do it alone, but I think made a substantial contribution to it. From the scientific point of view, I did some fundamental work in multidimensional scaling - I'm still doing fundamental work - and there's the whole Gifi system, which I've been doing since basically 1968. That's a large amount of work which has produced a huge number of students, who did their dissertations with me or with other people after I left, so that's a continuing tradition.
Is there a psychometrician or statistician who has really inspired your work?
Van de Geer was the first one to inspire me in that sense, and his geometrical approach to data analysis was different from what I eventually did, but it was inspiring anyway. In terms of direct influences, it was also the Bell Laboratory people, Kruskal and Carroll, and more distantly, Louis Guttman, who has an enormous body of work stretching over a long period of time, from the thirties to the 1970s, 1980s, of very good quality. I always enjoyed reading his work. Later on, he went a little bit off the rails.
When did he go off the rails?
In the 1980s I guess. That happens to a lot of people who are part of these 'schools', and have an enormous amount of influence on their students and insulate themselves. It happened to Guttman, it happened to Benzecri, it happened to Kalman, it happened to Hermann Wold, and that's not a very good development. I hope it didn't happen to me!
We'll find out! What do you believe is the most important work ever written in psychometrics, historically speaking?
I think that the general idea of latent structure analysis, with as its special cases IRT and factor analysis, is the most important idea to come out of psychometrics. It's not too strange to maintain that it originated in psychometrics. There was no psychometrics at the time, but it was developed by people who were doing what we now call psychometrics, and it has been developed to a large extent in psychometrics until it became a respectable and popular method in other disciplines as well.
What is still the biggest hurdle, or the biggest challenge, for psychometrics?
That would probably be the training or education that future psychometricians receive. I'm not entirely sure about the situation at other universities, but I know that at UCLA, there's some psychometric teaching and education, but Bengt Muthén can probably tell you more about that. And there's some training in psychology and Peter Bentler can tell you more about that, but it's fairly minimal. I also think that probably there's not enough contact, even now, between psychometrics centers and official academic statistics. Obviously we try to do something about that, but it has only been partially successful. It's easy to make joint appointments but it's more difficult to integrate teaching activities in different departments. So I think it's probably a challenge to keep things going in psychometrics. It's not a big field, it doesn't have a big journal, it's not a big society; it's marginal. Psychometrics should be much larger I think, if they capitalize on the fact that there's now data all over the place; the whole world is filled with data.
So psychometrics has the potential to be bigger.
I think so yes.
But it hasn't happened yet.
No, but as I said, I haven't really been involved in the Psychometric Society for quite a long time. Basically, I may not be up to date.
I think many people share the idea that the Psychometric Society could be bigger if there would be more exchanging of knowledge.
Yes.
So you're retired, but you're still doing work.
I'm producing more than I have for 30 years, 40 years. I'm working with the aforementioned Patricks, Mair and Groenen, on this new version of the Gifi project, so that's one thing. With the same two Patricks, I'm also working on the new multidimensional scaling programs. And everything we do these days consists of publications, many of them electronic and open source. And we're working on the programs in R, that's a relatively new way of making things, which is very convenient for me. Since I'm retired I don't really care whether something gets published or not. If people want to see something I wrote they know where to find it; they can Google me or look at my website. That's probably a suboptimal way of producing things if you need tenure somewhere, but it's quick.
It's perfect for you.
It's perfect for me and in a sense it's funny, because that's exactly the way I wrote things in 1968 when I started producing these never-ending series of internal reports at the data theory department, and I'm now producing a never-ending series of internal reports right here in Portland.
The circle comes back around. Thank you for this interview!
Albert Gifi (1990). Nonlinear Multivariate Analysis (Eds. W. J. Heiser, J. J. Meulman, G. van der Berg). New York: Wiley.
Meerling (1981). Methoden en technieken van psychologisch onderzoek, deel II. Data analyse en Psychometrie. Boom, Meppel.
Meerling (1980). Methoden en technieken van psychologisch onderzoek, deel I. Model, observatie en beslissing. Boom, Meppel.
Lisa D. Wijsen and Denny Borsboom (2021) Perspectives on Psychometrics: Interviews with 20 Past Psychometric Society Presidents, Psychometrika, volume 86, pages 327–343.