# vcaslib复现 **Repository Path**: satat/vcaslib-fx ## Basic Information - **Project Name**: vcaslib复现 - **Description**: 分布式大作业：复现论文实验 论文：Constant-Time Snapshots with Applications to Concurrent Data Structures（PPoPP 2021） - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2024-02-20 - **Last Updated**: 2024-11-23 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # 运行说明 ## 1.文件说明 📌ListReadMe.txt 介绍如何从0开始复现实验 📌jemalloc-5.2.1.tar.bz2 环境依赖 📌compile_all.out 编译命令的预期输出结果 📌run_all_tests.out 运行测试的预期输出结果 📌README.md 本文件 📌实验用脚本 为本次复现实验编写的脚本 📌运行结果 本次复现实验运行得到的结果 📌vcaslib 是论文提供的源码 - 📌LICENSE 源码版权声明 - 📌README.md 源码ReadMe，内容是论文引用 - 📌artifact 包含所有运行代码⭐ ## 2.复现环境 🛠系统：Linux ln152.para.bscc 3.10.0-1160.90.1.el7.x86_64 #1 SMP Thu May 4 15:21:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux （论文建议：Ubuntu 16.04.6 LTS） 🛠硬件：L分区/E分区：单节点48核，Intel Xeon Platimum 8163@2.5Ghz,192内存，IB高速网（论文建议：任何多核机器，至少有60gb主存） 🛠磁盘空间：0.5TB（论文建议所需磁盘空间为：46 MB） ## 3.运行说明 ### 🖊租赁云服务器（天玑算）： ①Linux version 3.10.0-1160.90.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Thu May 4 15:21:22 UTC 2023 ②上传文件夹：vcaslib里的artifact文件夹 和 jemalloc-5.2.1.tar.bz2 ### 🖊安装依赖： sudo apt-get install -y g++-9 sudo apt-get install -y openjdk-11-jdk sudo apt-get install -y python3-matplotlib sudo apt-get install -y numactl （若没有权限，需联系云计算平台相应技术人员配置，本次实验复现借助于天机算完成环境配置） ### 🖊下载并解压jemalloc-5.2.1.tar.bz2， ``` #若没有解压权限，执行以下两条命令，再使用命令解压 source /public1/soft/modules/module.sh module load rar/6.0.1 #若没有执行权限，用chmod命令 chmod +x autogen.sh #本实验指定jemalloc安装路径（如果是超级用户，不需要指定） ./autogen.sh --prefix=/public1/home/scf3007/soft/jemalloc ``` 依次输入以下三行命令 ./autogen.sh # step1 make # step2 sudo make install # step3 安装完成后，在 /usr/local下的lib、include、bin目录下会看到jemalloc的相关内容(参考：https://blog.csdn.net/SweeNeil/article/details/94648313) ### 🖊编译和运行 ./compile_all.sh # 编译 Java 和 C++ 的benchmarks ./run_all_tests.sh # 测试 Java 和 C++ 的benchmarks 编译命令的预期输出结果：compile_all.out 运行测试的预期输出结果：run_all_tests.out ### 🖊运行实验和生成图形 #若内存大于60G，可以直接运行以下代码 ./generate_graphs_from_paper_60.sh 本实验将所有绘图单独执行，运行脚本代码参考“本实验运行脚本sub.sh”或第4节 ### 🖊本实验运行脚本sub.sh sub.sh ``` #!/bin/bash #SBATCH -p queue #SBATCH -N 1 #SBATCH -n 1 #SBATCH -c 44 source /public1/home/scf3007/soft/env.sh export PATH=/public1/home/scf3007/soft/jemalloc/bin/:$PATH export LD_PRELOAD=/public1/home/scf3007/soft/jemalloc/lib/libjemalloc.so.2 # Figure 4a #python3 run_java_experiments_scalability.py 100000 3-2-95-0-0 [1,35,70,140] graphs/figure4a.png 5 5 120G # Figure 4b #python3 run_java_experiments_scalability.py 100000 30-20-50-0-0 [1,35,70,140] graphs/figure4b.png 5 5 120G # Figure 4c #python3 run_java_experiments_scalability.py 100000 30-20-50-1-1024 [1,35,70,140] graphs/figure4c.png 5 5 120G # Figure 4d #python3 run_java_experiments_scalability.py 100000000 3-2-95-0-0 [1,35,70,140] graphs/figure4d.png 5 5 120G # Figure 4e #python3 run_java_experiments_scalability.py 100000000 30-20-50-0-0 [1,35,70,140] graphs/figure4e.png 5 5 120G # Figure 4f #python3 run_java_experiments_scalability.py 100000000 30-20-50-1-1024 [1,35,70,140] graphs/figure4f.png 5 5 120G # Figures 4g and 4h 失败 #python3 run_java_experiments_rqsize.py 100000 36 36 [8,64,256,1024,8192,65536] graphs/figure4g4h.png 5 5 120G # Figures 5a and 5b 失败 python3 run_cpp_experiments_rqsize.py 100000 36 36 [8,64,256,1024,8192,65536] graphs/figure5a5b.png 5 5 # Figure 4j #python3 run_java_experiments_sorted.py [1,35,70,140] graphs/figure4j.png 10 5 120G # Figure 4k #python3 run_java_experiments_overhead.py 140 100000 100000000 graphs/figure4k.png 5 5 120G # Figure 4m #python3 run_java_experiments_multipoint.py 36 36 100000000 graphs/figure4m.png 5 5 120G #-------------- # Figure 5c 失败 #python3 run_cpp_experiments_memory_usage.py 100000 36 36 [8,64,256,1024,8192,65536] graphs/figure5c.png 5 5 # Figure 4i 失败 #python3 run_java_experiments_memory_usage.py 100000 36 36 [8,64,256,1024,8192,65536] graphs/figure4i.png 5 5 120G ``` 测试脚本：sub.sh （请按需修改使用核数/节点数/队列/运行命令等信息） 提交作业：sbatch sub.sh 查看作业情况：squeue 结束作业：scancel 作业号（作业号执行squeue即可查看到） 实时查看输出文件：tail -f 文件名 ### 🖊py程序说明 ``` run_java_experiments_scalable .py:可伸缩性实验(图4a-f) run_java_experiments_rqsize.py: Java范围查询大小实验(图4g-h) run_cpp_experiments_rqsize.py: c++范围查询大小实验(图5a-b) run_java_experiments_sorted.py:排序插入实验(图4j) run_java_experiments_overhead.py:开销VCAS(图4k) run_java_experiments_multipoint.py:多点查询的开销(图4m) run_cpp_experiments_memory_usage.py: c++内存使用情况(图5c) run_java_experiments_memory_usage.py: Java内存使用情况(图4i) ``` ## 4.运行结果 ### Figure 4a ±0.5h python3 run_java_experiments_scalability.py 100000 3-2-95-0-0 [1,35,70,140] graphs/figure4a.png 5 5 120G ![4a](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4a.png) ### Figure 4b ±0.5h python3 run_java_experiments_scalability.py 100000 30-20-50-0-0 [1,35,70,140] graphs/figure4b.png 5 5 120G ![4b](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4b.png) ### Figure 4c ±0.5h python3 run_java_experiments_scalability.py 100000 30-20-50-1-1024 [1,35,70,140] graphs/figure4c.png 5 5 120G ![4c](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4c.png) ### Figure 4d ±2.5h python3 run_java_experiments_scalability.py 100000000 3-2-95-0-0 [1,35,70,140] graphs/figure4d.png 5 5 120G ![4d](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4d.png) ### Figure 4e ±2.5h python3 run_java_experiments_scalability.py 100000000 30-20-50-0-0 [1,35,70,140] graphs/figure4e.png 5 5 120G ![4e](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4e.png) ### Figure 4f ±2.5h python3 run_java_experiments_scalability.py 100000000 30-20-50-1-1024 [1,35,70,140] graphs/figure4f.png 5 5 120G ![4f](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4f.png) ### Figures 4g and 4h 失败 python3 run_java_experiments_rqsize.py 100000 36 36 [8,64,256,1024,8192,65536] graphs/figure4g4h.png 5 5 120G ### Figures 5a and 5b 失败 python3 run_cpp_experiments_rqsize.py 100000 36 36 [8,64,256,1024,8192,65536] graphs/figure5a5b.png 5 5 运行失败，输出日志为缺少特定文件。 ### Figure 4j ±0.5h python3 run_java_experiments_sorted.py [1,35,70,140] graphs/figure4j.png 10 5 120G ![4j](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4j.png) ### Figure 4k ±1h python3 run_java_experiments_overhead.py 140 100000 100000000 graphs/figure4k.png 5 5 120G ![4k](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4k.png) ### Figure 4m 1h python3 run_java_experiments_multipoint.py 36 36 100000000 graphs/figure4m.png 5 5 120G ![4m](%E8%BF%90%E8%A1%8C%E7%BB%93%E6%9E%9C/figure4m.png) -------------------------- ### Figure 5c 失败 python3 run_cpp_experiments_memory_usage.py 100000 36 36 [8,64,256,1024,8192,65536] graphs/figure5c.png 5 5 运行失败，输出日志中的信息提示，缺少特定文件。 ### Figure 4i 失败（0.5h但未出图） python3 run_java_experiments_memory_usage.py 100000 36 36 [8,64,256,1024,8192,65536] graphs/figure4i.png 5 5 120G 运行半个多小时后，报错：TypeError: _init_() got an unexpected keyword argument "basex" ## 5.版权说明 代码来自论文：Constant-Time Snapshots with Applications to Concurrent Data Structures（PPoPP 2021） 非盈利侵删 Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. PPoPP ’21, February 27-March 3, 2021, Virtual Event, Republic of Korea © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-8294-6/21/02. . . $15.00 https://doi.org/10.1145/3437801.3441602 Code licenses: MIT License ## 6.原论文完整引用 Yuanhao Wei, Naama Ben-David, Guy E. Blelloch, Panagiota Fatourou, Eric Ruppert, and Yihan Sun. 2021. Constant-time snapshots with applications to concurrent data structures. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP '21). Association for Computing Machinery, New York, NY, USA, 31–46. https://doi.org/10.1145/3437801.3441602 ## 7.致谢 感谢天机算的计算资源 感谢天玑智算-江山 和 并行科技应用技术支持工程师5的耐心帮助。