convert: Fix detection of LLAMA2

In recent downloads of LLAMA2 dataset the norm_eps is set to 1e-06, this
leads to convert.py erroneously considering the model to be LLAMA1 and
setting the context to 2k tokens.

Fix it by extending the existing hack to also check for the 1e-06 value.

Author: Nikolay Borisov

convert.py

@@ -245,12 +245,13 @@ def loadHFTransformerJson(model: LazyModel, config_path: Path) -> Params:
     @staticmethod
     def loadOriginalParamsJson(model: LazyModel, config_path: Path) -> Params:
         config = json.load(open(config_path))
+        norm_eps = config["norm_eps"]
 
         # hack to determine LLaMA v1 vs v2 vs CodeLlama
         if config.get("rope_theta") == 1000000:
             # CodeLlama
             n_ctx = 16384
-        elif config["norm_eps"] == 1e-05:
+        elif norm_eps == 1e-05 or norm_eps == 1e-06:
             # LLaMA v2
             n_ctx = 4096
         else: