https://advances.sciencemag.org/content/5/1/eaat7550.full
Therein we read, for example:
"the Graduate Record Examination (GRE) Quantitative, Verbal, and Physics Subject Tests do not predict completion as effectively admissions committees presume. Significant associations with completion were found for undergraduate GPA in all models and for GRE Quantitative in two of four studied models; GRE Physics and GRE Verbal were not significant in any model"
There followed a further commentary ('The Usefulness of GRE Scores') in the Physics Today Readers' Forum ( January, 2020, p. 10) in which those statistics were challenged by Michael B. Weissman, who wrote:
"Although that paper uses data provided by many physics departments, I found some serious statistical flaws in its analysis. Contrary to its conclusions, proper statistical analysis of even the incomplete published features of the data indicates that an equal weight sum of the quantitative and physics GREs is somewhat better than grade point average at predicting who will graduate."
Moreover, this is reasonable given that we already know undergrad GPAs are subject to outrageous inflation, which I (and others) have attributed to student evaluations of professors. See e.g.
Thus, the use of a standardized test like the GRE - namely its physics and quantitative sections- would be ideal to avoid the implicit specter of subjective grade inflation incurred at so many universities. Oh, some profs may well deny this occurs, but having seen it up close and personal in different departments, I can assert that is bollocks It exists, and it applies to both ethnic minority groups and others - to any group of students who may have an axe to grind and a willingness to use it in an evaluation. (Which I will cover in a post next week, on how a number of physics departments are getting ready to phase them out..)
Prof. Weissman goes on to advise that if we are to take papers like that cited (in Science Advances) seriously, we need to have better transparency and also, "greater statistical literacy in handling the data." I would suggest in regard to the latter, the researchers look more closely at selection effects in the assorted departments, i.e. the use of student evaluations and how these have plausibly affected the undergraduate success embodied in the GPAs. This would require going through different class inputs, say for General Physics, and comparing the students evals vs. what actual GPAs were earned, and also whether other issues manifested. (Class, lab attendance, degree of class participation, results of tests on specific labs - after they are completed, 'pop' quiz results, project quality, etc.)
Another pertinent point made by Weissman is:
"Systematic uncertainties in estimating the effects of using GREs in admission decisions would remain even after a proper analysis of more complete data,as is typical for any attempt to estimate causal parameters from observational data."
In this regard Weissman advises the APS (American Physical Society) asking for departments to "volunteer in a randomized control trial".
This would entail some departments being assigned "GRE-aware" admissions and others, "GRE-blind" admissions.
But I would warrant the former would find much better results especially for the GRE-Q. Why? First, because physics is a quantitative science so it stands to reason that success - irrespective of ethnic background or gender- must depend a great deal on quantitative ability. For example, consider the following question which I posted back in August, 2013, on looking at GRE quantitative examples. I'd argue any serious physics student about to graduate from a proper physics department (NO grade inflation tolerated!) ought to be able to ace it:
-----
The graph below shows the radiocarbon C-14 excess over C-12 over a 2,000 year period.
In general, C14 is produced in the upper atmosphere via the impact –interaction with high energy cosmic rays, say from galactic sources. Solar activity in turn modulates the intensity of these cosmic rays via the action of the heliosphere's magnetic field which deflects a fraction of the intense cosmic ray flux and other harmful interstellar radiation.
When the Sun is more active, the heliosphere will be stronger, shielding the Earth from more intense cosmic rays the effect of which is to reduce the C14 produced in the Earth’s upper atmosphere. Conversely, when the Sun is less active then the shield is weaker and more intense cosmic rays penetrate to our upper atmosphere yielding more C14 produced.
To conform with observed solar activity the plot (from P.E. Damon :'The Solar Output and Its Variation', The University of Colorado Press, Boulder, 1977) is such that increasing radiocarbon (C14) is downward and indicated with (+). The deviations in parts per thousand are shown relative to an arbitrary 19th century reference level.
Questions:
1. The increasing C14/C12 ratio, i.e. from left to right on the graph probably arises from:
(A) Injection of anthropogenic CO2 into the Earth's atmosphere.
(B) The result of the known, slow decrease in the strength of the Earth’s magnetic moment.
(C) Increased cosmic ray fluxes on Earth
(D) Increased radiocarbon production on Earth resulting from (C)
(E) All the above
2. The sharp upward spike at the modern end of the curve (e.g. from about 1850 on) , representing a marked drop in relative radiocarbon, is probably due to:
(A) Anthropogenic causes—the onset of increased population and the Industrial Age
(B) The burning of low radiocarbon fossil fuels, such as coal and oil
(C) The systematic burning off of the world’s forests for agriculture.
(D) Processes (B) and (C) only
(E) All the processes: A, B and C
To conform with observed solar activity the plot (from P.E. Damon :'The Solar Output and Its Variation', The University of Colorado Press, Boulder, 1977) is such that increasing radiocarbon (C14) is downward and indicated with (+). The deviations in parts per thousand are shown relative to an arbitrary 19th century reference level.
Questions:
1. The increasing C14/C12 ratio, i.e. from left to right on the graph probably arises from:
(A) Injection of anthropogenic CO2 into the Earth's atmosphere.
(B) The result of the known, slow decrease in the strength of the Earth’s magnetic moment.
(C) Increased cosmic ray fluxes on Earth
(D) Increased radiocarbon production on Earth resulting from (C)
(E) All the above
2. The sharp upward spike at the modern end of the curve (e.g. from about 1850 on) , representing a marked drop in relative radiocarbon, is probably due to:
(A) Anthropogenic causes—the onset of increased population and the Industrial Age
(B) The burning of low radiocarbon fossil fuels, such as coal and oil
(C) The systematic burning off of the world’s forests for agriculture.
(D) Processes (B) and (C) only
(E) All the processes: A, B and C
--------------------------------------------------------------
Likewise, the case can be made that the GRE Physics subject test is a good indicator of whether the newly minted grad is ready to move on to graduate school. Consider the following sampling of questions (from a total of 100 - to be completed in 3 hrs.) extracted from one subject test:
The diagram (Fig. 1) shows a resting cylinder of weight W. The coefficient for static friction for all surfaces is 1/3. The applied force for P = 2W
A) d = r/3 B) d = d/2 C) d = r/ 4 D) d = 2d/ 5 E) d = 3r/ 5
62) The vertical reaction force at point A is:
63) The vertical reaction force at point B is:
A) 3W…..B) 2.7 W……C) 1.5 W…..D) 2.1 W….E) 2.5 W
64) The horizontal reaction force at point A is:
A) 1.5 W…..B) 2.1 W…. C) 0.9 W……D) 1.2 W……E) 1.8 W
65) The horizontal reaction force at point B is:
A) 1.5 W…..B) 02.1 W…..C) 0.9 W….D) 1.2 W…..E) 1.8 W
66) If the frequency of the sound produced by a siren increases from 400 cps to 1200 cps while the wave amplitude remains constant, the ratio of the intensity of the 1200 cps wave to the 400 cps wave is:
A) 1: 1…..B) 1: 3…..c) 3: 1……D) 1: 9 ….E) 9:1
67) A stream of water leaving the nozzle of a hose horizontally 4 feet above the ground strikes the ground at a horizontal distance of 20 feet. The velocity of the water when leaving the nozzle, in feet per second, is:
A) 20…..B) 30……C) 40……D) 60……E) 80
68) A wire of length L has a cross-sectional area A and Young’s modulus Y. The force constant should be equal to:
A) YA/ L……B) A/ YL……C) Y……D) L/YA …E) Y/A
69) A battery is connected to a resistor causing a current of 0.6 amps to flow. When an additional 4.0 ohm resistor is added in series to the circuit, the current drops to 0.5 amp. The emf of the battery is:
A) 5 volts….B) 6 volts…..C) 12 volts…..D) 4 volts…..E) 24 volts
70) A galvanometer having an armature coil with a resistance of 10 ohms requires 0.01 amps for a full scale deflection. To convert this galvanometer to a voltmeter which will give a full scale deflection when the voltage is 120 volts, a coil in series must be added that will have a resistance in ohms of:
A) 40…..B) 120…..C) 1,200……D) 1,210……E) 11,900
71) It is known from quantum mechanics that a particle in a one dimensional potential well (such as shown in the diagram) can exist in a number of energy states. Imagine an electron confined between the boundaries x and x + Dx, where Dx is 0.5 Angstroms.
Approximately, what is the uncertainty in the x-component of the momentum of the electron ?
72) If DP x is the uncertainty in the x –component of the momentum of the electron, what is the uncertainty in the y-component of the momentum of the electron ?
73) Consider a free particle confined between two impenetrable walls at x and x + L. What is the probability according to classical physics that the particle will be found between x and x + L/3 if no other information is given?
74) What is the probability according to quantum mechanics that the particle in its lowest energy state will be found between x and x + L/3?
75) What is the probability according to quantum mechanics that the particle in the second lowest energy state will be found between x and x + L/3?
-----------------------------------------------------------------------
Again, as per Weissman's points on transparency, this is testing pure physics aptitude, not how good the student was at manipulating a prof to get a better grade, or how Machiavellian his threats were if he didn't (say in terms of a teacher evaluation). It is therefore great for objective assessment, say in readiness to do the Ph.D., given there is no external or subjective nonsense to confuse the issue. I would argue, indeed, that such a GRE Physics test ought to be substituted as a final exam for all physics grads, in whatever university. Such a test definitely would eliminate the noise from profs being potentially threatened with poor evaluations, say if they don't deliver the inflated grade. (Or tempted to grade too soft, or with an absurd 'curve')
As for predicting Ph.D. completion I believe it is absurd to try to use the GRE or any other gauge, say like undergrad GPA (especially for the reasons already given) because such academic markers ignore or omit real world influences, i.e. in departmental politics, poor thesis supervision, personality clashes with advisers, etc. Those causative factors then tend to be underplayed because they are, well, messy, and you won't find many departments ready or willing to hang out their 'dirty linen'. But make no mistake it's there and has an impact on Ph.D. completion.
Finally, to complete the Ph.D. the student - of whatever background (gender, ethnicity) will have to pass his Ph.D. comprehensive examinations. That will entail solving a lot of difficult traditional problems! Generally, there will be six exams in six subject areas: classical mechanics, thermodynamics, statistical mechanics, mathematical physics, quantum physics, and electromagnetic theory. These will generally be four hours each. One example problem from the mechanics exam is shown below:
It is clear to me, given what is needed to complete the Ph.D., that disallowing the GRE physics and subject exams as pre-qualifiers for entry into such programs is counter-productive - and doesn't redound to the benefit of aspiring students (or their advisers) irrespective of whether they are "minority" students or not.
It is clear to me, given what is needed to complete the Ph.D., that disallowing the GRE physics and subject exams as pre-qualifiers for entry into such programs is counter-productive - and doesn't redound to the benefit of aspiring students (or their advisers) irrespective of whether they are "minority" students or not.
No comments:
Post a Comment