add llama2 models; remove some absolute paths

README.md

 System for generating DEFT 2023 outputs from LLMs
 =================================================
 
-* Update 2023-09-26: add code to finetune and perform inference with LLaMa2
+* Update 2023-09-26: add code to finetune and perform inference with LLaMa2 (performance as good as ChatGPT)
 
 The DEFT'23 shared task consists in answering pharma exam MCQs. This system converts the questions and possible answers to prompts and uses LLMs to generate answers.
 The approach is described in our [paper](http://talnarchives.atala.org/CORIA-TALN/CORIA-TALN-2023/479307.pdf). It ranked 1st at the shared task.
 This repository contains scripts to generate prompts, run off-the-shelf models and finetune the LLaMa models. It also contains the LoRA weights for the finetuned models. 
 
 This repository uses git LFS for large files.
-Use 'git lfs install' before cloning to retrive the binary files.
+Use 'git lfs clone...' for cloning with the binary files.
 
 Install:
 ```
-pip install -r requirements.txt
+pip install -r requirements.txt # for llama1 
+pip install -r requirements.llama2-freeze.txt # for llama2
 ```
 
 Note that bitsandbytes may need to be recompiled to support your cuda version.
+Note that llama2 was finetuned with Python/3.10.10 and CUDA/11.6 on a single A100-80 GPU
 
 See RESULTS for the exact match results on the dev.
 See runs for how to generate runs.
+See trains for llama2 finetuning runs.
 
-Note that external APIs require API keys. Please rename api_keys.template.py to api_keys.py and set keys you need inside.
+Note that external APIs require API keys. Rename api_keys.template.py to api_keys.py and set keys you need inside.
 
 Please cite the follwing paper:
 ```

llama2_models/llama-2-13b-deft-comp/README.md

+---
+library_name: peft
+---
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: float16
+### Framework versions
+
+
+- PEFT 0.5.0

llama2_models/llama-2-13b-deft-comp/adapter_config.json

+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "/storage/raid1/corpora/llama2-weights/convert/llama-2-13b-hf/",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 4,
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}
\ No newline at end of file

llama2_models/llama-2-13b-deft/README.md

+---
+library_name: peft
+---
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: float16
+### Framework versions
+
+
+- PEFT 0.5.0

llama2_models/llama-2-13b-deft/adapter_config.json

+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "/storage/raid1/corpora/llama2-weights/convert/llama-2-13b-hf/",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 4,
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}
\ No newline at end of file

llama2_models/llama-2-70b-deft-comp/README.md

+---
+library_name: peft
+---
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: float16
+### Framework versions
+
+
+- PEFT 0.5.0

llama2_models/llama-2-70b-deft-comp/adapter_config.json

+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "/storage/raid1/corpora/llama2-weights/convert/llama-2-70b-hf/",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 4,
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}
\ No newline at end of file

llama2_models/llama-2-70b-deft/README.md

+---
+library_name: peft
+---
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: float16
+### Framework versions
+
+
+- PEFT 0.5.0

llama2_models/llama-2-70b-deft/adapter_config.json

+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "/storage/raid1/corpora/llama2-weights/convert/llama-2-70b-hf/",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 4,
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}
\ No newline at end of file

llama2_models/llama-2-7b-deft-comp/README.md

+---
+library_name: peft
+---
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: float16
+### Framework versions
+
+
+- PEFT 0.5.0

llama2_models/llama-2-7b-deft-comp/adapter_config.json

+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "/storage/raid1/corpora/llama2-weights/convert/llama-2-7b-hf/",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 4,
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}
\ No newline at end of file

llama2_models/llama-2-7b-deft/README.md

+---
+library_name: peft
+---
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: float16
+### Framework versions
+
+
+- PEFT 0.5.0

llama2_models/llama-2-7b-deft/adapter_config.json

+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "/storage/raid1/corpora/llama2-weights/convert/llama-2-7b-hf/",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 4,
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}
\ No newline at end of file

requirements.llama2-freeze.txt

+accelerate==0.23.0
+aiohttp==3.8.5
+aiosignal==1.3.1
+appdirs==1.4.4
+async-timeout==4.0.3
+attrs==23.1.0
+bitsandbytes==0.41.1
+certifi==2023.7.22
+charset-normalizer==3.2.0
+click==8.1.7
+cmake==3.27.5
+datasets==2.14.5
+dill==0.3.7
+docker-pycreds==0.4.0
+filelock==3.12.4
+fire==0.5.0
+frozenlist==1.4.0
+fsspec==2023.6.0
+gitdb==4.0.10
+GitPython==3.1.37
+huggingface-hub==0.17.2
+idna==3.4
+Jinja2==3.1.2
+Levenshtein==0.21.1
+lit==16.0.6
+MarkupSafe==2.1.3
+mpmath==1.3.0
+multidict==6.0.4
+multiprocess==0.70.15
+networkx==3.1
+numpy==1.26.0
+nvidia-cublas-cu11==11.10.3.66
+nvidia-cuda-cupti-cu11==11.7.101
+nvidia-cuda-nvrtc-cu11==11.7.99
+nvidia-cuda-runtime-cu11==11.7.99
+nvidia-cudnn-cu11==8.5.0.96
+nvidia-cufft-cu11==10.9.0.58
+nvidia-curand-cu11==10.2.10.91
+nvidia-cusolver-cu11==11.4.0.1
+nvidia-cusparse-cu11==11.7.4.91
+nvidia-nccl-cu11==2.14.3
+nvidia-nvtx-cu11==11.7.91
+packaging==23.1
+pandas==2.1.1
+pathtools==0.1.2
+peft==0.5.0
+protobuf==3.20.0
+psutil==5.9.5
+pyarrow==13.0.0
+python-dateutil==2.8.2
+pytz==2023.3.post1
+PyYAML==6.0.1
+rapidfuzz==3.3.0
+regex==2023.8.8
+requests==2.31.0
+safetensors==0.3.3
+scipy==1.11.2
+sentry-sdk==1.31.0
+setproctitle==1.3.2
+six==1.16.0
+smmap==5.0.1
+sympy==1.12
+termcolor==2.3.0
+tokenizers==0.13.3
+torch==2.0.1
+tqdm==4.66.1
+transformers==4.33.2
+triton==2.0.0
+trl==0.7.1
+typing_extensions==4.8.0
+tzdata==2023.3
+urllib3==2.0.5
+wandb==0.15.11
+xxhash==3.3.0
+yarl==1.9.2