Perhaps it’s a semantics issue. However, forensically speaking it’s validation not testing. Call it what you will, but in my world you cannot use software for forensically related purposes without validation. It’s very easy to do to a search.
Ann H. Ross, Ph.D., D-ABFA
I think this is where we differ about validation.
What you and Ann describe is "testing" or "debugging". For me, that is not validation, and it is definitely part of the coding process and needs to be learned and practiced. But without a formal mathematical proof, the validation tends to be data and use case specific.
So taking a piece of code "validated" in the manner and then applying to a completely different type of data and/or use cases is not a good thing (an example would be trying to use UMAP instead of PCA in a morphospace reconstruction. it is valid, but incorrect). That's why I said the onus is on the user. On Sunday, January 5, 2025 at 8:19:45 PM UTC-8 Soda, Kenneth Prof. wrote:
Hello Everyone,
As someone who spends a lot of time thinking about scientific programming, both as a researcher and an educator, I think you are all correct. Programming is a valuable skill set that does empower researchers to have greater flexibility in how they analyze their
data and that can contribute to greater transparency in the scientific process.
However, my two cents is that a strong scientific programmer should also be a strong scientific
validator. I encourage my students to have a healthy skepticism towards the code that they write and to independently validate that the code is operating as it was intended to operate (e.g., through simple test cases that the coder completes using pen-and-paper).
Actually, I would say more of my research time is spent validating my code than actually writing it. Some research institutions have entire teams fully devoted to code validation, and some journals, such as the Journal of Statistical Computing, ask their reviewers
to review not only the written manuscript but also whether the code is correct (i.e., to validate the code).
Even if you do not know how to code, validation is something that you can consider looking for when deciding whether to use software, be that software open-source code or a GUI. Does the author or company show that the code has gone through the validation process?
Did the author submit a peer-reviewed publication where the software's correctness was validated? Consider putting the software that you use through the same thought process as you would an article that you read because software, like an article, is a scientific
contribution.
In the end, I think we all want the same thing, that our programs provide reliable research results, just as we would want reliable researcher results from a wet lab or field experiment.
That's how I see it anyway,
K. James Soda (He/Him/His)
##############################################
Assistant Professor of Mathematics and Data Science
Department of Mathematics and Statistics
Quinnipiac University
Hamden, CT
##############################################
CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
Dear Ann,
Not sure what you mean by not validated? What is the validation of a t-test function in R or any other software library? Validation is the responsibility of the person using the tools, not the developers. Open-source tools make this validation for more
simple, since if you have any concern you can look under the hood. Evaluate the code line by line and then if you find an issue you can easily take it up with the developers.
In closed-source software this is almost near impossible. I would argue this is a far more superior method of "validation" then to appeal to the authority.
On Sunday, January 5, 2025 at 6:46:15 PM UTC-8 Ann Ross wrote:
Hi All,
The one thing to keep in mind that all this coding is important but not validated. One needs to trust the results and if not validated leaves a lot of questions. This is from a forensic perspective.
Ann
Ann H. Ross, Ph.D., D-ABFA
Ann H. Ross, Ph.D., D-ABFA
On Jan 5, 2025, at 7:48 PM, 'Adams, Dean [EEOB]' via Morphmet <[email protected]> wrote:
I completely understand that R and other coding approaches can seem daunting. But I agree 100% with Murat, and encourage you and others to steer into the wind! Coding is empowering!
First, a bit of R-coding allows you to improve your data manipulation and curation. This helps with scientific repeatability, compliance with journal requirements (which increasingly require one to submit curated data and scripts to a
public repository), and basically enhances and encourages open-source science.
But more importantly, coding empowers you as a scientist. By moving to R, you remove yourself from the restraints that exist with canned software, whose options are necessarily limited by the buttons and options that the user has available
to point-and-click. In essence, while prepackaged software is easy to use, it limits thinking and creativity by restricting
one’s analysis to those options that the software happens to have.
The unfortunate outcome of such canned software is that our GM literature becomes filled with many studies of similar analyses: not because those biological topics are inherently interesting per se, but because that is what the software
happens to allow. This is the analytical version of what one of my mentors (Larry Slobodkin) once called ‘intellectual painting by numbers.’ I seriously hope that our field can do better now in the 21st century.
My message: learning a bit of code breaks this cycle, and frees one to investigate the questions that one actually wishes to explore, not just those for which canned software has already provided. I strongly encourage you (and others)
to learn a bit of R, Python, or some other language so that your creative science is not restricted!
Best of luck in your journey.
Dean
Dr. Dean C. Adams
Distinguished Professor of Evolutionary Biology
Department of Ecology, Evolution, and Organismal Biology
Iowa State University
https://faculty.sites.iastate.edu/dcadams/
phone:
515-294-3834
From:
[email protected] <[email protected]>
On Behalf Of Murat Maga
Sent: Sunday, January 5, 2025 12:29 PM
To: Morphmet <[email protected]>
Subject: Re: [MORPHMET2] MorphoJ ProcrustesANOVA
A quick comment: Instead of trying to work this with PAST, I highly encourage you to spend that time trying to do the same in R using geomorph, or other shape analysis libraries. Yes, it will probably take longer, yes it will be somewhat
bumpy road initially. But you will be much better set for the next challenge. Graphical user interface applications are good up to a point (for common tasks). And I am telling you as someone who is developing UI based morphometrics analysis.
Most often in biology, you will have to customize your analysis to the specific question you are trying to answer. This is best done via scripting in a flexible programming environment (whether that's R or Python or some other language
is irrelevant). There are tons of resources and a very supportive community to help you with your challenges. Going forward, all fields of biology will be more computational not less, and the sooner you start warming up to the idea it will be better for your
career.
Your future self will thank you for that decision.
Thanks everyone... I suspected this was the case but wanted to be sure I wasn't missing anything. I'm going to look into using PAST first, since I'm not the most comfortable in R.
On Saturday, January 4, 2025 at 5:34:57 AM UTC-8 Adams, Dean [EEOB] wrote:
the tests are parametric in MorphoJ as far as I know.
If you're analyzing symmetry/asymmetry, the 'trick' I used needs to be reworked and may not work but it should be possible to do an equivalent analysis in one of the morphometric packages (morpho, geomorph, not sure about momocs).
I am sure a skilled R coder will be able to suggest better ways.
Dear morphmet members,
I expect this will be a relatively easy question.
Does the Procrustes ANOVA in MorphoJ use permutations? I don't see it as an option, though I see it in regression and in Matrix correlation.
I expect that since there isn't an option for it, then it is not a permutation-based test, but it seems so odd that it wouldn't be.
I can't find it in the documentation, and I'm convinced it must be there and I'm missing it. If there is something that discusses this, could you kindly point me to it.
--
You received this message because you are subscribed to the Google Groups "Morphmet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
[email protected].
To view this discussion visit
https://groups.google.com/d/msgid/morphmet2/549e6df3-2eaf-46cc-9b34-f30f5caf2777n%40googlegroups.com.
--
--
You received this message because you are subscribed to the Google Groups "Morphmet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
[email protected].
--
You received this message because you are subscribed to the Google Groups "Morphmet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
[email protected].
To view this discussion visit
https://groups.google.com/d/msgid/morphmet2/5a1598a7-96b9-4cd4-9a1e-ec45c61fcc64n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Morphmet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
[email protected].
--
You received this message because you are subscribed to the Google Groups "Morphmet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
[email protected].
--
You received this message because you are subscribed to the Google Groups "Morphmet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion visit https://groups.google.com/d/msgid/morphmet2/75903bc0-fb1c-4752-82db-9eef850aea18n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Morphmet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion visit https://groups.google.com/d/msgid/morphmet2/E90DBF76-51BF-4F47-8033-C1549C0078F6%40ncsu.edu.
|
