| --- |
| license: cc-by-nc-4.0 |
| language: |
| - en |
| pipeline_tag: conversational |
| tags: |
| - fair-use |
| - llama2 |
| - ggml |
| --- |
| |
| # Llama2 Movie Character Finetuned 7F Quantized |
|
|
| Quantized by llama.cpp (https://github.com/ggerganov/llama.cpp) |
| Either use by llama.cpp or llama cpp python wrappers |
|
|
| with ctransformers: |
|
|
| ``` |
| #pip install ctransformers |
| from ctransformers import AutoModelForCausalLM |
| |
| llm = AutoModelForCausalLM.from_pretrained("llama2-7f-fp16-ggml-q4.bin", |
| model_type='llama', |
| gpu_layers=100, # you can use less (like 20) if you have less gpu ram |
| max_new_tokens=50, |
| stop=["###","##"], |
| threads=4) # 4 cpu threads to limit cpu |
| ``` |
|
|
| with llama-cpp-python: |
| ``` |
| |
| # !CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python |
| |
| model_path="llama2-7f-fp16-ggml-q4.bin" |
| llm = Llama(model_path=model_path,n_gpu_layers=100) |
| output = llm("### Context: talks friendly### History: ### Morpheus: I challenge you to battle of words!### Gerald:", max_tokens=32, stop=["#","sierpeda"], echo=True) |
| |
| ``` |
|
|
| ## Prompt format |
| ``` |
| prompt = f"Context: {overall_context}### History: {previous_dialogue} ### {character_1}: {question}### {character_2}:" |
| |
| ``` |
|
|
|
|
|
|
| ## Some Fun dialogues |
|
|
| ## Geralt of Witcher vs Ig-11 of Mandalorian |
|
|
| context= "on a cruise ship going to Mars from Earth" |
|
|
| #Gerald(Geralt Witcher, do not know why I named him incorrect) asks Ig-11 (Mandalorian) |
| initial_text = What do you think about AI? |
| |
| ``` |
| Gerald : What do you think about AI? |
| Ig-11 : I've worked with them for more than half of my operational cycle. |
| Gerald : So, how long has the ship been here? |
| Ig-11 : 765 years (33 days) since last port. |
| Gerald : So this is the fourth time since I joined the ship that we've had a murderer on board? |
| Ig-11 : My master does not believe in coincidences... He believes everything happens for a reason. The universe has a plan, and it includes me being here at the same time as you. So, what do you think about AI? |
| Gerald : I'm not sure yet... I mean, they work on Earth, and maybe beyond. |
| Ig-11 : You see? It is all so simple! The humans are going to Mars; we are bringing them with us. |
| ``` |
| |
| |
| ## Batman vs Tony Stark |
| context= "tries to save the world" |
| |
| initial_text = "I am richer than you!" |
|
|
| ``` |
| Batman : I am richer than you! |
| Tony Stark : My car goes 100mph! |
| Batman : My car can go at least that fast, and it's not even suped up. everybody in the car. It didn't even have a license plate! It was so loud I couldn't hear myself think. It felt like we were gonna get killed any second... |
| Tony Stark : You need to chill out, Bruce. You can be such a bitch sometimes.~Tony Stark |
| Batman : You don't get it do you? It's not about justice. It's about power.~Bruce Wayne |
| Tony Stark : You know the one thing that really gets under my skin? When someone shoots himself in the foot so bad, he ends up putting on a hobble~Tony Stark |
| Batman : "Let me tell you something. Something I've learned to my great cost..."~Batman |
| Tony Stark : "You make it sound like a disease." |
| ``` |
|
|
| ## Darth Vader vs James T. Kirk |
|
|
| ``` |
| Vader : I challenge you to battle of words! |
| Kirk : What are your terms? |
| Vader : The Emperor's terms! |
| Kirk : I challenge you to the ultimate game, a verbal battle of wit and wile. surely you know that no mere mortal can stand against my brilliance. |
| Vader : I accept your challenge. The terms I propose is, the winner shall have the pleasure of killing the loser! |
| Kirk : Deal! But you will be disappointed. Though my words are powerful and sharp, they lack the strength to kill. |
| Vader : You speak truthfully. My vocabulary lacks power. It is but a fraction of what it was 10 years ago at the height of my powers. |
| Kirk : I've never heard you speak like this before. |
| ``` |
