ValueError: The matrix should be symmetric on .matrix files made with HiC-Pro

[Balthasar-eu]

When I try to manually do ice normalization on matrix files from HiC-Pro ice fails with the following error:

Traceback (most recent call last):
  File "/home/balthasar/.local/bin/ice", line 95, in <module>
    counts, bias = iced.normalization.ICE_normalization(
  File "/home/balthasar/.local/lib/python3.8/site-packages/iced/normalization/__init__.py", line 66, in ICE_normalization
    is_symetric_or_tri(X)
  File "/home/balthasar/.local/lib/python3.8/site-packages/iced/utils/validation.py", line 13, in is_symetric_or_tri
    raise ValueError("The matrix should be symmetric")
ValueError: The matrix should be symmetric

The matrix file in question was made by downloading an allValidPairs file from Github and then building the matrix file with the build_matrix script from HiC-Pro.

When issuing the ice command like this:

ice output.matrix

The above error appears after a while.

[WangxiaowuLab]

I also meet the problem,could you tell me how to solve the problem?
Thank you very much!

[NelleV]

Hello,
which version of ice and numpy are you using? This may be due to numerical instabilities in the checks.
Cheers,
N

[WangxiaowuLab]

Hello, which version of ice and numpy are you using? This may be due to numerical instabilities in the checks. Cheers, N

The numpy version is 1.21.1 and HiC-Pro is 3.0.0

[NelleV]

Do you mind sharing raw matrix with me so that I have a look at what is going on? Many thanks, Nelle

[WangxiaowuLab]

The matrix file and bed file  are in supplement. Thank you!

…

------------------ 原始邮件 ------------------ 发件人: "hiclib/iced" ***@***.***>; 发送时间: 2021年12月17日(星期五) 凌晨0:41 ***@***.***>; ***@***.******@***.***>; 主题: Re: [hiclib/iced] ValueError: The matrix should be symmetric on .matrix files made with HiC-Pro (#98) Do you mind sharing raw matrix with me so that I have a look at what is going on? Many thanks, Nelle — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: ***@***.***>

[WangxiaowuLab]

It is succcessful when I run ice with the matrix from hic-pro,but it fail when run with the other matrix file(ValueError:The matrix should be symmetric).I have checked the two matrix file format but do not find difference.

[NelleV]

I think you've attached the files to the email: unfortunately, github doesn't support this. Can you plus upload them directly through the interface?
Many thanks,
Nelle

[WangxiaowuLab]

I think you've attached the files to the email: unfortunately, github doesn't support this. Can you plus upload them directly through the interface? Many thanks, Nelle

Hello!
I have solved the problem.Use the commond 'awk '$1>$2{print $0}' matrixfile ' and I find there are some lines that first number bigger than second .When I delete those line,the ice work well.
Thanks for your help!

[Balthasar-eu]

I am in holiday right now and don't have access to the computer I processed the files originally. I can check if that solves the issues in January. Also I see that I confused the the origin of the files in the original post. I did not download them from github, but GEO.

Just to clarify though, I did not solve the problem. Our own hic files were processed from fastq and the ice normalization in the pipeline worked without error, so we used outside input files that did not need ice normalization to avoid the problem.