Hugging Face announced Falcon 180B, the most powerful open source LLM ever. It rivals Google’s Palm 2… And it has no guardrails
Hugging Face has unveiled Falcon 180B, the largest open source Large Language Model that it claims can match Google’s Palm 2 cutting-edge AI in terms of performance. Additionally, it lacks guardrails that would prevent it from producing hazardous or unsafe outputs.
Falcon 180B Performs at a State-of-the-Art Level
“State of the art” refers to something that is operating at its peak potential, matching or even outperforming the greatest available example at the time.
When researchers declare that an algorithm or huge language model has achieved state-of-the-art performance, it’s a major thing.
Hugging Face says the same thing about Falcon 180B.
Falcon 180B outperforms earlier open source models and “rivals” Google’s Palm 2 in terms of natural language processing performance.
And those aren’t just boasts.
Data support Hugging Face’s assertion that Falcon 180B competes with Palm 2.
According to the statistics, Falcon 180B outperforms Llama 270B, the previous most potent open source model, on a number of tests used to gauge an AI model’s potency.
Falcon 180B even outperforms OpenAI’s GPT-3.5.
The testing data also shows that Falcon 180B performs at the same level as Google’s Palm 2.
Screenshot of Performance Comparison
The announcement explained:
“Falcon 180B is the best openly released LLM today, outperforming Llama 2 70B and OpenAI’s GPT-3.5…
Falcon 180B typically sits somewhere between GPT 3.5 and GPT4 depending on the evaluation benchmark…”
FThe announcement continues to hint that further tweaking the model by users could boost performance even further.
Minor technical concerns that complicate indexing include internal links to outdated URLs that have been updated with a category structure that cause 301 redirects.
Dataset Used To Train Falcon 180B
This dataset is made up entirely of web content that was gathered from the open source Common Crawl, a freely accessible web dataset.
The dataset is next filtered and subjected to a deduplication procedure (the removal of redundant or duplicate data) to enhance the quality of what is left.
With the screening, the researchers hope to get rid of automated spam, boilerplate, repetitive content, copied content, and information that doesn’t reflect natural language.
Falcon 180B Has Zero Guardrails
Falcon 180B is notable in that no alignment tuning has been applied to keep it from producing negative or hazardous output and nothing has been done to stop it from making up information and simply lying.
As a result, the model can be adjusted to produce results that Google and OpenAI’s products are unable to produce.
This is detailed in the announcement’s limits section.
Hugging Face advises:
“Limitations: the model can and will produce factually incorrect information, hallucinating facts and actions.
As it has not undergone any advanced tuning/alignment, it can produce problematic outputs, especially if prompted to do so.”
Commercial Use Of Falcon 180B
Hugging Face allows commercial use of Falcon 180B.
However it’s released under a restrictive license.
Those who wish to use Falcon 180B are encouraged by Hugging Face to first consult a lawyer.
Falcon 180B Is Like A Starting Point
Lastly, the model hasn’t undergone instruction training, which means that it has to be trained to be an AI chatbot.
So it’s like a base model that needs more to become whatever users want it to be. Hugging Face also released a chat model but it’s apparently a “simple” one.
Hugging Face explains:
“The base model has no prompt format. Remember that it’s not a conversational model or trained with instructions, so don’t expect it to generate conversational responses—the pretrained model is a great platform for further finetuning, but you probably shouldn’t directly use it out of the box.
The Chat model has a very simple conversation structure.”
