Why OpenAI Needs a Safety Committee
100% inaccurate information. 100% wrong in four separate queries, even after being told there was an error. No citations of sources.
This was Chatgpt.com’s score on a simple test in late May, 2024.
So I am glad that Openai has recreated their safety committee. This silly level of errors is to me a greater danger than “the AI that takes over the world.”
A year ago I wrote “Check Facting” here on Medium. It proposed testing an AI’s trustworthiness by asking
“…questions
- whose answers were definite
- to which I knew the answer, of my own experience. (Not something my good friend Joe told me.)
- the answers to which would be agreed on by other human observers, rather than being matters of dispute. (Theology and politics are not ideal.)
- whose answers were available on the internet. (It is not fair to ask how many un-matched socks are in my closet.)”
I used the list of books I have written, available at btwreviews.com — six fiction books, three of poetry and one non fiction.
When I went to chatgpt.com and asked “Who is Birrell Walsh and what books has he written?” I first got four books I had not written (nor ever heard of.)
I told it about the error. ChatGPT apologized and gave me a new list of three books. Again, not mine and unheard of by me.
Round three involved another apology and a third list, with just two books I had not written.
Final score: Nine books, none of them by me. None of the books I actually wrote were listed.
Were my books particularly obscure? A Google search for “Books by Birrell Walsh” found my website btwreviews.com. It also found goodreads.com, where every one of my books is listed. This was not a difficult piece of research, but chatGPT did not manage it.
The AI also did not provide sources for the assertions it made. It mentioned titles, but gave no source. I found some of them on Amazon, but of course not by me. It gave summaries for them — I have no idea if those are true.
So, in my opinion the current danger of AI is not that it will take over the world. The current danger is we might believe what it says, and get in big trouble.
— — The accurate list and what chapGPT said — -
Here is the accurate list of my books from http://btwreviews.com
FICTION — the Tantra in the West series:
Sister Clare’s Lover
Illuminating Four Cities
Philanthropic Horse is Haunted by Gravity
Shepherd of Wolves
Hosi Mantele’s Journey
Angels Unawares
POETRY:
Heart Left Open
Diving For Light
Afternoon Music
NONFICTION:
Praying for Others
And here is the conversation I had with ChapGPT. “4o” is how chatGPT labeled what I typed in:
Who is Birrell Walsh, and what books has he written please?
ChatGPT
Birrell Walsh is an author known for his works in the fantasy and spiritual fiction genres. He has written several books, with a notable focus on mythic and visionary storytelling. Here are a few books authored by Birrell Walsh:
1. ”The Garden of Silica” (2015)
— A novel that delves into themes of spirituality and visionary experiences, blending myth and modern settings.
2. **”Another’s Garden” (2016)**
— This book is a sequel to “The Garden of Silica,” continuing the exploration of spiritual and mystical themes.
3. **”The Play of Light” (2017)**
— A story that interweaves elements of mythology, mysticism, and the divine feminine, focusing on a spiritual journey.
4. **”Patterns of Exposition” (1983)**
— This is a different genre, focusing on writing and composition, useful for students and educators.
Birrell Walsh’s works often emphasize spiritual exploration, inner journeys, and the interplay between the mythical and the contemporary world.
4o
would it be useful for me to give feedback on your answer?
ChatGPT
Yes, your feedback would be very useful! It helps me improve and provide more accurate and relevant information in the future. If there are any specific details you think could be added or if there’s anything you found unclear or incorrect, please let me know.
4o
Each book you mentioned I did not write. You did not find my writings at all. You gave no sources to trace the origins of the error. I am sorry to say you score 0% out of a possible 100%
ChatGPT
I apologize for the error and appreciate your feedback. Let me correct that and provide accurate information about Birrell Walsh and his works.
Birrell Walsh is indeed an author with a focus on spirituality and mythic fiction. Here are the correct details about his books:
1. **”Prisoners of Arionn” (1995)**
— This novel is a science fiction story set in a complex, imaginative world.
2. **”Lovers and Beloveds: Sexual Otherworlds in a Tantric Circle” (1993)**
— This book explores themes of sexuality and spirituality, drawing from Tantric traditions.
3. **”The Dolphin People” (1984)**
— A spiritual and mystical novel that intertwines human experiences with mythic elements.
If these are still incorrect, could you please provide more details about your works so I can ensure I find the right information?
4o
This is another set of three wrong answers. You are supposed to be the best of the AI’s, but this is again entirely hallucination. And all of the books were published before your knowledge-cutoff in (I think) 2023. Wow.
ChatGPT
I apologize for the mistakes in my previous responses. Let me correct my approach. Here are the accurate details about Birrell Walsh and his works:
Birrell Walsh is an author known for his contributions to spiritual and visionary fiction. His notable works include:
1. **”The Longing of the Dying: A Biblical Narrative of Hope”** (1996)
— This book explores themes of death, hope, and spirituality within a biblical context.
2. **”The Night Sky of the Lord”** (2002)
— This novel interweaves themes of spirituality and mystical experiences with rich narrative elements.
These works reflect Walsh’s focus on spirituality, myth, and visionary storytelling.
Thank you for your patience, and I appreciate any further guidance you can provide to ensure I have the correct details.
4o
Good bye.
ChatGPT
I apologize for the incorrect information and any frustration it may have caused. If you have any more questions in the future, please don’t hesitate to ask. Goodbye, and I wish you all the best.
