1,模型训练:pytorch/paddle/tensorflow+pandas+numpy+scipy+os,有时候也会用sklearn
-
pytorch
https://pytorch.org/tutorials/recipes/recipes/Captum_Recipe.html
https://pytorch.org/docs/stable/generated/torch.unsqueeze.html#torch.unsqueeze
分布式训练: https://mp.weixin.qq.com/s/5yTalo0KICnZ2otfs6KJSw
https://www.zhihu.com/question/436008648 -
tensorflow
显存释放:https://blog.csdn.net/qq_41653523/article/details/110353184 -
pandas
- https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html
- 数据类型转换
data['列索引'].astype(type) data['索引'].apply(自定义函数) ``
-
sklearn
计算AUC(Area Under Curve):from sklearn.metrics import roc_auc_score
2,绘图:matplotlib,visio,sklearn(可绘制混淆矩阵)
- matplotlib
https://matplotlib.org/api/_as_gen/matplotlib.pyplot.subplot2grid.html#matplotlib.pyplot.subplot2grid
https://www.cnblogs.com/tensorzhang/p/14576189.html (常规用法总结)
3,模型结构展示:tensorwatch+jupyter lab,torch.utils.tensorboard, tensorboardX(但是tensorwatch画出来的图都是零碎的,也不知道问题在哪。tensorboard则是直接不显示graph-路径不要有中文)
-
jupyter lab
https://www.cnblogs.com/feffery/
https://jupyter.org/documentation -
tensorwatch
https://github.com/microsoft/tensorwatch
https://cloud.tencent.com/developer/article/1449697 -
tensoboard
https://www.tensorflow.org/tensorboard/get_started -
tensorboardX
https://github.com/lanpa/tensorboardX
4,特征追踪:
-
pytorch+captum
https://captum.ai/
https://captum.ai/tutorials/
https://gilberttanner.com/blog/interpreting-pytorch-models-with-captum
https://www.kaggle.com/zhangkaiyu/notebookd049a46a87/edit -
scikit-learn+ELI5
https://eli5.readthedocs.io/en/latest/
5,图像数据处理:
- 数据增强:imgaug
https://imgaug.readthedocs.io/en/latest/source/examples_basics.html - opencv
http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_gui/py_image_display/py_image_display.html - PyDicom
一个用于处理DICOM格式文件的Python包,可以处理包括如医学图像(CT等)、报告等。
https://zhuanlan.zhihu.com/p/59413289