• AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf'

    关注 码龄 粉丝数 原力等级 -- 被采纳 被点赞 采纳率 普通网友 2024-06-14 19:53 采纳率: 0% 浏览 17 首页/ 编程语言 / AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf' python自然语言处理语言模型 对chatglm-6b-int4进行ptuning训练，按照官方教程运行bash train.sh后无法进行训练，报错AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf' 报错内容：-106/14/2024 16:42:03 - WARNING - main - Process rank: -1, device: cuda:0, n_gpu: 1distributed training: False, 16-bits training: False06/14/2024 16:42:03 - INFO - main - Training/evaluation parameters Seq2SeqTrainingArguments(_n_gpu=1,adafactor=False,adam_beta1=0.9,adam_beta2=0.999,adam_epsilon=1e-08,auto_find_batch_size=False,bf16=False,bf16_full_eval=False,data_seed=None,dataloader_drop_last=False,dataloader_num_workers=0,dataloader_pin_memory=True,ddp_backend=None,ddp_bucket_cap_mb=None,ddp_find_unused_parameters=None,ddp_timeout=1800,debug=[],deepspeed=None,disable_tqdm=False,do_eval=False,do_predict=False,do_train=True,eval_accumulation_steps=None,eval_delay=0,eval_steps=None,evaluation_strategy=no,fp16=False,fp16_backend=auto,fp16_full_eval=False,fp16_opt_level=O1,fsdp=[],fsdp_config={'fsdp_min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},fsdp_min_num_params=0,fsdp_transformer_layer_cls_to_wrap=None,full_determinism=False,generation_config=None,generation_max_length=None,generation_num_beams=None,gradient_accumulation_steps=16,gradient_checkpointing=False,greater_is_better=None,group_by_length=False,half_precision_backend=auto,hub_model_id=None,hub_private_repo=False,hub_strategy=every_save,hub_token=,ignore_data_skip=False,include_inputs_for_metrics=False,jit_mode_eval=False,label_names=None,label_smoothing_factor=0.0,learning_rate=0.02,length_column_name=length,load_best_model_at_end=False,local_rank=-1,log_level=passive,log_level_replica=warning,log_on_each_node=True,logging_dir=output\qa-chatglm-6b-int4-pt-v2\runs\Jun14_16-42-03_DESKTOP-68C74C8,logging_first_step=False,logging_nan_inf_filter=True,logging_steps=10,logging_strategy=steps,lr_scheduler_type=linear,max_grad_norm=1.0,max_steps=100,metric_for_best_model=None,mp_parameters=,no_cuda=False,num_train_epochs=3.0,optim=adamw_hf,optim_args=None,output_dir=output\qa-chatglm-6b-int4-pt-v2,overwrite_output_dir=True,past_index=-1,per_device_eval_batch_size=1,per_device_train_batch_size=1,predict_with_generate=True,prediction_loss_only=False,push_to_hub=False,push_to_hub_model_id=None,push_to_hub_organization=None,push_to_hub_token=,ray_scope=last,remove_unused_columns=True,report_to=[],resume_from_checkpoint=None,run_name=output\qa-chatglm-6b-int4-pt-v2,save_on_each_node=False,save_safetensors=False,save_steps=50,save_strategy=steps,save_total_limit=None,seed=42,sharded_ddp=[],skip_memory_metrics=True,sortish_sampler=False,tf32=None,torch_compile=False,torch_compile_backend=None,torch_compile_mode=None,torchdynamo=None,tpu_metrics_debug=False,tpu_num_cores=None,use_ipex=False,use_legacy_prediction_loop=False,use_mps_device=False,warmup_ratio=0.0,warmup_steps=0,weight_decay=0.0,xpu_backend=None,)D:\Anaconda\envs\DL\lib\site-packages\datasets\load.py:2552: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.You can remove this warning by passing 'token=' instead. warnings.warn([INFO|configuration_utils.py:667] 2024-06-14 16:42:04,933 >> loading configuration file D:\langchain-model\chatglm-6b-int4\config.json[INFO|configuration_utils.py:667] 2024-06-14 16:42:04,935 >> loading configuration file D:\langchain-model\chatglm-6b-int4\config.json[INFO|configuration_utils.py:725] 2024-06-14 16:42:04,935 >> Model config ChatGLMConfig { "_name_or_path": "D:\\langchain-model\\chatglm-6b-int4", "architectures": [ "ChatGLMModel" ], "auto_map": { "AutoConfig": "configuration_chatglm.ChatGLMConfig", "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration" }, "bos_token_id": 130004, "eos_token_id": 130005, "gmask_token_id": 130001, "hidden_size": 4096, "inner_hidden_size": 16384, "layernorm_epsilon": 1e-05, "mask_token_id": 130000, "max_sequence_length": 2048, "model_type": "chatglm", "num_attention_heads": 32, "num_layers": 28, "pad_token_id": 3, "position_encoding_2d": true, "pre_seq_len": null, "prefix_projection": false, "quantization_bit": 4, "quantization_embeddings": false, "torch_dtype": "float16", "transformers_version": "4.30.0", "use_cache": true, "vocab_size": 130528} [INFO|tokenization_utils_base.py:1821] 2024-06-14 16:42:04,937 >> loading file ice_text.model[INFO|tokenization_utils_base.py:1821] 2024-06-14 16:42:04,937 >> loading file added_tokens.json[INFO|tokenization_utils_base.py:1821] 2024-06-14 16:42:04,937 >> loading file special_tokens_map.json[INFO|tokenization_utils_base.py:1821] 2024-06-14 16:42:04,937 >> loading file tokenizer_config.json[INFO|modeling_utils.py:2575] 2024-06-14 16:42:05,145 >> loading weights file D:\langchain-model\chatglm-6b-int4\pytorch_model.bin[INFO|configuration_utils.py:577] 2024-06-14 16:42:06,454 >> Generate config GenerationConfig { "_from_model_config": true, "bos_token_id": 130004, "eos_token_id": 130005, "pad_token_id": 3, "transformers_version": "4.30.0"} --- Logging error ---Traceback (most recent call last): File "C:\Users\HP/.cache\huggingface\modules\transformers_modules\chatglm-6b-int4\quantization.py", line 19, in from cpm_kernels.kernels.base import LazyKernelCModule, KernelFunction, round_up File "D:\Anaconda\envs\DL\lib\site-packages\cpm_kernels_init.py", line 1, in from . import library File "D:\Anaconda\envs\DL\lib\site-packages\cpm_kernels\library_init.py", line 2, in from . import cuda File "D:\Anaconda\envs\DL\lib\site-packages\cpm_kernels\library\cuda.py", line 7, in cuda = Lib.from_lib("cuda", ctypes.WinDLL("nvcuda.dll")) File "D:\Anaconda\envs\DL\lib\site-packages\cpm_kernels\library\base.py", line 63, in from_lib ret = Lib(name) File "D:\Anaconda\envs\DL\lib\site-packages\cpm_kernels\library\base.py", line 45, in init lib_path = windows_find_lib(self.__name) File "D:\Anaconda\envs\DL\lib\site-packages\cpm_kernels\library\base.py", line 39, in windows_find_lib return lookup_dll(lib_name) File "D:\Anaconda\envs\DL\lib\site-packages\cpm_kernels\library\base.py", line 16, in lookup_dll for name in os.listdir(path):NotADirectoryError: [WinError 267] Ŀ¼▒▒▒▒▒▒Ч▒▒: 'C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\Anaconda\envs\DL\lib\logging_init.py", line 1083, in emit msg = self.format(record) File "D:\Anaconda\envs\DL\lib\logging_init.py", line 927, in format return fmt.format(record) File "D:\Anaconda\envs\DL\lib\logging_init.py", line 663, in format record.message = record.getMessage() File "D:\Anaconda\envs\DL\lib\logging_init.py", line 367, in getMessage msg = msg % self.argsTypeError: not all arguments converted during string formattingCall stack: File "D:\langchain-model\chatglm-6b-int4\ChatGLM-6B-main\ChatGLM-6B-main\ptuning\main.py", line 433, in main() File "D:\langchain-model\chatglm-6b-int4\ChatGLM-6B-main\ChatGLM-6B-main\ptuning\main.py", line 127, in main model = AutoModel.from_pretrained(model_args.model_name_or_path, config=config, trust_remote_code=True) File "D:\Anaconda\envs\DL\lib\site-packages\transformers\models\auto\auto_factory.py", line 479, in from_pretrained return model_class.from_pretrained( File "D:\Anaconda\envs\DL\lib\site-packages\transformers\modeling_utils.py", line 2675, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "C:\Users\HP/.cache\huggingface\modules\transformers_modules\chatglm-6b-int4\modeling_chatglm.py", line 1061, in init self.quantize(self.config.quantization_bit, self.config.quantization_embeddings, use_quantization_cache=True, empty_init=True) File "C:\Users\HP/.cache\huggingface\modules\transformers_modules\chatglm-6b-int4\modeling_chatglm.py", line 1424, in quantize from .quantization import quantize, QuantizedEmbedding, QuantizedLinear, load_cpu_kernel File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "", line 850, in exec_module File "", line 228, in _call_with_frames_removed File "C:\Users\HP/.cache\huggingface\modules\transformers_modules\chatglm-6b-int4\quantization.py", line 46, in logger.warning("Failed to load cpm_kernels:", exception)Message: 'Failed to load cpm_kernels:'Arguments: (NotADirectoryError(20, 'Ŀ¼▒▒▒▒▒▒Ч▒▒'),)[INFO|modeling_utils.py:3295] 2024-06-14 16:42:07,190 >> All model checkpoint weights were used when initializing ChatGLMForConditionalGeneration. [WARNING|modeling_utils.py:3297] 2024-06-14 16:42:07,190 >> Some weights of ChatGLMForConditionalGeneration were not initialized from the model checkpoint at D:\langchain-model\chatglm-6b-int4 and are newly initialized: ['transformer.prefix_encoder.embedding.weight']You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.[INFO|modeling_utils.py:2927] 2024-06-14 16:42:07,206 >> Generation config file not found, using a generation config created from the model config.Load kernel : D:\langchain-model\chatglm-6b-int4\quantization_kernels.soNo set_num_threads() found in kernel.Setting CPU quantization kernel threads to 10Using quantization cacheApplying quantization to glm layersQuantized to 4 bitLoad kernel : D:\langchain-model\chatglm-6b-int4\quantization_kernels.soNo set_num_threads() found in kernel.Setting CPU quantization kernel threads to 10Running tokenizer on train dataset: 0%| | 0/255 [00:00

    3条回答 默认 最新

    • 专家-赤兔[在线] 全栈领域优质创作者 2024-06-14 19:53
      关注

      引自免费微信小程序：皆我百晓生

      The error message indicates that there's an issue with loading or accessing a specific function int4WeightExtractionHalf from a module related to the cpm_kernels library, which is likely being used for the quantization process of the chatglm-6b-int4 model. The root cause appears to be a failure to find and load the necessary CUDA library (nvcuda.dll), which leads to the cpm_kernels library not being properly initialized, hence the NoneType error when trying to access its functions.

      To resolve this issue, you need to ensure that the CUDA toolkit is correctly installed and configured in your environment. Here are the steps you can follow:

      1. Verify CUDA Installation: Ensure that CUDA is installed on your system. You can check this by running CUDA sample codes or using commands like nvcc --version. If CUDA is not installed, download and install it from the official NVIDIA website: NVIDIA CUDA Toolkit.

      2. Environment Variable Path: Check if the CUDA library paths are included in your system's environment variables. Specifically, verify that the directory containing nvcuda.dll (usually something like C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vX.Y\bin) is in your system's PATH. If not, add it.

      3. Reinstall cpm_kernels Library: Sometimes reinstalling the library can help resolve issues due to corrupted or outdated files. Try uninstalling and then reinstalling the cpm_kernels library:

        pip uninstall cpm_kernels
pip install cpm_kernels

      4. Check for Conflicting Libraries: Ensure there are no conflicting versions of CUDA libraries installed. Sometimes having multiple versions of CUDA installed can lead to loading the wrong DLLs. Clean up any redundant CUDA installations.

      5. Restart Your Environment: After making changes to the environment variables or installing/uninstalling packages, restart your Python environment or IDE to ensure that the changes take effect.

      If after following these steps the issue persists, consider checking the compatibility of the CUDA version with the version of PyTorch or the specific requirements of the cpm_kernels library. Sometimes, specific versions of CUDA are required for certain libraries to work correctly.

      In case you're still facing issues, you might want to reach out to the maintainers of the cpm_kernels library or the chatglm-6b-int4 model repository for more specialized support. They may have encountered similar issues and could provide more detailed guidance based on their experience with the project.

      展开全部

      评论 编辑记录
    • 阿里嘎多学长 2024-06-14 19:53
      关注

      以下内容由CHATGPT及阿里嘎多学长共同生成、有用望采纳：

      根据您提供的错误信息，这个问题似乎与模型的量化处理有关。AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf' 这个错误表明在尝试访问一个不存在的属性 int4WeightExtractionHalf。这通常发生在尝试调用一个未正确初始化或未导入的模块或对象时。

      以下是一些可能的解决步骤：

      1. 确认环境配置：确保您的Python环境配置正确，所有依赖库都是最新版本，并且与chatglm-6b-int4模型兼容。

      2. 检查量化模块：错误发生在调用cpm_kernels库的过程中。您需要确保这个库已经正确安装，并且可以被您的代码访问到。

      3. 检查CUDA和cuDNN：由于您在使用CUDA相关的操作，确保您的CUDA和cuDNN库是最新的，并且与您的PyTorch版本兼容。

      4. 检查代码：检查quantization.py文件中的extract_weight_to_half函数，确保所有必要的依赖都已经正确导入。

      5. 环境路径问题：错误信息中提到了路径问题，NotADirectoryError: [WinError 267]，这表明代码尝试访问的路径不存在或不正确。检查您的系统路径设置，确保所有必要的库文件都可以被正确找到。

      6. 调试：尝试逐步执行代码，查看在调用int4WeightExtractionHalf之前，kernels对象是否已经被正确初始化。

      7. 查看官方文档或论坛：有时候，类似的问题可能已经在官方文档或者论坛上被讨论过，查看是否有相关的issue或者解决方案。

      8. 联系作者或社区：如果上述步骤都不能解决问题，您可能需要联系模型的作者或者在相关的技术社区发帖求助。

      由于我无法直接运行代码或访问您的系统环境，以上步骤仅供参考。如果需要更具体的代码实现或解决方案，您可能需要提供更多的上下文信息，或者直接联系模型的维护者。

      至于参考资料，您可以查看chatglm-6b-int4模型的官方文档或者GitHub仓库，通常这些资源会提供安装指南、常见问题解答和社区支持链接。如果模型是基于Hugging Face的Transformers库，那么您也可以查看Transformers的官方文档来获取更多帮助。

      评论 编辑记录
    • Kwan的解忧杂货铺@新空间代码工作室 2024-06-14 19:54
      关注

      晚上好🌙🌙🌙
      本答案参考ChatGPT-3.5

      是的，我理解了您的问题。这是一个关于Python的调试错误。我在读取命令行参数时遇到了这个问题。

      AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf'

      这是因为您没有正确地传递模型的参数。对于ChatGLM，int4WeightExtractionHalf是一个超参数，应该由模型的构建者在调用ChatGLM的API时提供。

      解决此问题的方法是在创建模型实例时指定这个超参数：

      from pytorch_gluonnlp import ChatGLM

model = ChatGLM(
    architectures=["ChatGLMModel"],
    initiliaze="configuration_chatglm.ChatGLMConfig",
    int4_weight_extraction_half="your_value_here"
)

      在这个例子中，将int4_weight_extraction_half替换为“your_value_here”。确保你已经从数据集中加载了一些具有这种类型的权重的样本，并将其设置为你想要添加到ChatGLM中的值。

      希望这能帮到您！如果您还有其他问题，欢迎随时向我提问。

      评论
    编辑
    预览

    报告相同问题？

  • 相关阅读:
    Pytorch框架学习记录4——数据集的使用（torchvision.dataset）
    Zookeeper入门
    [附源码]JAVA毕业设计仟侬堂茶具网站（系统+LW）
    ubuntu无法使用man
    96. 不同的二叉搜索树
    【linux】倒计时小程序
    多路查找树
    解决Flutter应用程序的兼容性问题
    Spring Boot中如何优雅地实现异步调用？
    Spring源码解析——IOC之bean 的初始化
  • 原文地址：https://ask.csdn.net/questions/8118793