From 73cc6097eec4959e3e3f9c4b35a13e66268c9624 Mon Sep 17 00:00:00 2001 From: anchuanxu Date: Fri, 8 Jul 2022 16:08:48 +0800 Subject: [PATCH 1/2] add ospp2022 vio doc --- ...56\351\242\230\346\214\207\345\215\227.md" | 36 +++++++ ...27\346\263\225\347\256\200\344\273\213.md" | 93 +++++++++++++++++++ 2 files changed, 129 insertions(+) create mode 100644 "ospp2022/vio/ROS_SIG\351\227\256\351\242\230\346\214\207\345\215\227.md" create mode 100644 "ospp2022/vio/\345\237\272\344\272\216ROS\346\216\245\345\217\243\347\232\204VIO\347\256\227\346\263\225\347\256\200\344\273\213.md" diff --git "a/ospp2022/vio/ROS_SIG\351\227\256\351\242\230\346\214\207\345\215\227.md" "b/ospp2022/vio/ROS_SIG\351\227\256\351\242\230\346\214\207\345\215\227.md" new file mode 100644 index 0000000..ce6e004 --- /dev/null +++ "b/ospp2022/vio/ROS_SIG\351\227\256\351\242\230\346\214\207\345\215\227.md" @@ -0,0 +1,36 @@ +# ROS SIG问题指南 + + + +### （1）ROS包移植步骤 + +ROS基础软件包以及其上下游的依赖包移植，是open-Euler社区ROS SIG要完成的主线工作。这部分工作的意义就是让ROS及其相关软件在open-Euler操作系统上成功稳定的运行起来，扩展社区的上层应用生态，使得社区的软件和硬件环境可以有更多的扩展性。 + +- step 1 + +检查open-Euler源上是否已经存在了相关的软件包，检查ROS-SIG已移植的软件包列表，是否已经存在该包，避免重复工作。 + +https://gitee.com/openeuler/community/tree/master/sig/sig-ROS + +- step 2 + +接下来就要将ROS包进行本地构建，具体的操作文档可以参考社区贡献者已有教程文档： + +[OpenEuler21.03 本地打包ros_comm并验证](https://gitee.com/openeuler/ros/blob/master/document/OpenEuler21.03%20%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ros_comm%E5%B9%B6%E9%AA%8C%E8%AF%81.md) + +[OpenEuler21.03 ROS1 OBS打包指南](https://gitee.com/openeuler/ros/blob/master/document/OpenEuler21.03%20ROS1%20OBS%E6%89%93%E5%8C%85%E6%8C%87%E5%8D%97.md) + +[OpenEuler21.03 ROS2 OBS打包指南](https://gitee.com/openeuler/ros/blob/master/document/OpenEuler21.03%20ROS2%20OBS%E6%89%93%E5%8C%85%E6%8C%87%E5%8D%97.md) + +需要注意的是可能要对齐依赖库的版本问题，Python版本问题，以及CI门禁的相关问题。有关社区的CI门禁，请参考: + +[门禁功能指导手册](https://www.openeuler.org/zh/blog/zhengyaohui/2022-03-21-ci_guild.html) + +### （2）问题论坛 + +如果在开源的工作中遇到了困难，可以前往社区的互动平台，查找是否有相关的解决方案。社区博客：https://www.openeuler.org/zh/interaction/blog-list/ + +同时，社区也有丰富的文档供你学习，比如open-Euler 21.03的安装：https://docs.openeuler.org/zh/docs/21.03/docs/Quickstart/quick-start.html + +友情提示：社区内的文档偶尔也存在部分错误，需要自己亲自尝试并探索解决方案。 + diff --git "a/ospp2022/vio/\345\237\272\344\272\216ROS\346\216\245\345\217\243\347\232\204VIO\347\256\227\346\263\225\347\256\200\344\273\213.md" "b/ospp2022/vio/\345\237\272\344\272\216ROS\346\216\245\345\217\243\347\232\204VIO\347\256\227\346\263\225\347\256\200\344\273\213.md" new file mode 100644 index 0000000..bfff803 --- /dev/null +++ "b/ospp2022/vio/\345\237\272\344\272\216ROS\346\216\245\345\217\243\347\232\204VIO\347\256\227\346\263\225\347\256\200\344\273\213.md" @@ -0,0 +1,93 @@ +# 基于ROS接口的VO/VIO算法简介 + +### （1）SVO1.0算法 + +SVO（Semi-Direct Monocular Visual Odometry）半直接法视觉里程计，是苏黎世大学机器人感知组的克里斯蒂安.弗斯特于2014年ICRA会议上发表， 并在github上开源：https://github.com/uzh-rpg/rpg_svo。 + +所谓半直接是指通过对图像中的特征点图像块进行直接匹配来获取相机位姿，而不像直接匹配法那样对整个图像使用直接匹配。 虽然semi-direct方法使用了特征，但它的思路主要还是通过direct method来获取位姿，这和feature-method不一样。同时，semi-direct方法和direct method不同的是它利用特征块的配准来对direct method估计的位姿进行优化。 通俗点说，就是结合了特征点法和直接法的视觉里程计。 + +算法优点： + +- 稀疏直接法，运行速度非常快； +- 轻量化，可运行在嵌入式程序中，适配硬件广泛； + +缺点： + +- 算法在平视场景容易丢失跟踪，且无法恢复； +- 对光照环境有一定要求，在运动速度过快等条件下极容易丢失； + +### （2）SVO2.0算法 + +2021年7月14日苏黎世大学SVO2.0开源，论文：https://rpg.ifi.uzh.ch/docs/TRO17_Forster-SVO.pdf。 + +它是机器人和感知小组 (RPG) 开发的最新版本的半直接视觉里程计 (SVO)。如 SVO 论文 (TRO-17) 中所述，SVO 是作为一个快速且多功能的视觉前端而诞生的。从那时起，通过各种研究和工业项目整合了不同的扩展。 SVO Pro 支持不同的相机型号、主动曝光控制、基于滑动窗口的后端以及带闭环的全局束调整。 + +SVO2.0特点： + +- 视觉里程计：最新版本的 SVO，支持单目或立体设置中的透视和鱼眼/折反射相机。它还包括主动曝光控制； +- 视觉惯性里程计：SVO 前端 + 视觉惯性滑动窗口优化后端（由 OKVIS 修改）； +- 视觉惯性 SLAM：SVO 前端 + 视觉惯性滑动窗口优化后端 + 全局捆绑调整地图（使用 iSAM2）。借助 iSAM2，全球地图实时更新，并用于以帧速率进行定位； +- 具有闭环的视觉惯性 SLAM：通过 DBoW2 将闭环集成到全局束调整中。姿势图优化也被包括在内，作为全局捆绑调整的轻量级替代品。 + +![v102_gm.gif](https://github.com/uzh-rpg/rpg_svo_pro_open/raw/master/doc/images/v102_gm.gif) + +### （3）ORB_SLAM1-3算法 + +ORB_SLAM 是西班牙 Zaragoza 大学的 Raúl Mur-Arta 编写的视觉 SLAM 系统。 它是一个完整的 SLAM 系统，包括视觉里程计、跟踪、回环检测，是一种完全基于稀疏特征点的单目 SLAM 系统，同时还有单目、双目、RGBD 相机的接口。其核心是使用 ORB (Orinted FAST and BRIEF) 作为整个视觉 SLAM 中的核心特征。经过数年的发展，ORB_SLAM也已经发展了三代： + +开源代码： + +- https://github.com/raulmur/ORB_SLAM； +- https://github.com/raulmur/ORB_SLAM2； +- https://github.com/UZ-SLAMLab/ORB_SLAM3； + +ORB_SLAM是一种基于特征匹配的单目SLAM系统，该算法带有回环和重定位功能，所以前端的VO和后端的回环检测构成了整体的SLAM系统。ORB_SLAM2则在ORB_SLAM的基础上扩展到了单目、双目、RGB-D相机全部支持，包含地图复用、回环和重定位功能，其中轻量级的定位模式，利用VO跟踪未建图的区域，并与地图点匹配，允许零漂移定位。ORB_SLAM3则将相机阵营扩展到了针孔相机和鱼眼相机，并且融合了惯性导航单元，构建了一个紧耦合的VIO系统，鲁棒性有了巨大的提升。第二个创新点是多地图系统，它依赖于一种新的位置识别方法和改进版本的回环检测。这两点保证算法在弱视觉的场景下也能成功运行。 + +demo:https://www.youtube.com/watch?v=HyLNq-98LRo&feature=youtu.be + +### （4）DSO和LDSO算法 + +DSO（Direct Sparse Odometry），是慕尼黑工业大学（Technical University of Munich, TUM）计算机视觉实验室的雅各布.恩格尔（Jakob Engel）博士，于2016年发布的一个视觉里程计方法。DSO属于稀疏直接法的视觉里程计。它不是完整的SLAM，因为它不包含回环检测、地图复用的功能。因此，它不可避免地会出现累计误差，尽管很小，但不能消除。DSO目前开源了单目实现。 + +DSO是少数使用纯直接法（Fully direct）计算视觉里程计的系统之一。相比之下，SVO[2]属于半直接法，仅在前端的Sparse model-based Image Alignment部分使用了直接法，之后的位姿估计、bundle adjustment，则仍旧使用传统的最小化重投影误差的方式。而ORB-SLAM2，则属于纯特征法，计算结果完全依赖特征匹配。从方法上来说，DSO是新颖、独树一帜的。 + +LDSO则是在DSO基础上加入了闭环检测模块，算法的鲁棒性和精确度得到了进一步的提升。 + +开源代码： + +- https://github.com/JakobEngel/dso； +- https://github.com/tum-vision/LDSO； + +### （5）LSD算法 + +LSD-SLAM是一种新颖、直接的单目SLAM技术:它不使用关键点，而是直接对图像强度进行操作，用于跟踪和映射。相机跟踪使用直接图像对齐，而几何估计形式的半密集深度地图，通过过滤许多像素立体比较获得。然后，我们构建关键帧的Sim(3)姿态图，这允许构建经过比例漂移修正的大规模地图，包括循环闭包。LSD-SLAM可以在CPU上实时运行，甚至可以在现代智能手机上运行。 + +开源代码：https://github.com/tum-vision/lsd_slam； + +LSD-SLAM系统整个过程分为三大模块，第一个模块是Tracking,第二个模块是Depth Map Estimation, 第三个模块是Map Optimization + +[![image](https://github.com/DroidAITech/vslam-evaluation/raw/master/LSD/Installation%20documentation/images/LSD.png)](https://github.com/DroidAITech/vslam-evaluation/blob/master/LSD/Installation documentation/images/LSD.png) + +1.跟踪部分连续地跟踪摄像机获取的图像，即估计相对于当前的关键帧的刚体变换位姿，其中初始值为前一帧的位姿。 +2.深度地图估计部分将跟踪的帧用于精化或者替代当前关键帧。通过对像素小基线立体的比较滤波和空间规则化来精化深度。如果相机移动过快，通过将已存在的邻近的关键帧的点投影到当前帧来初始化一个新的关键帧。 +3.地图优化部分将作为当前跟踪参考帧的关键帧整合到全局地图中，该关键帧的深度地图将不再进行精化。此外，使用尺度感知的直接图像配准方法估计相对于临近关键帧的相似变换，实现回路闭合和尺度漂移的检测。 + +### （6）VINS-Mono算法 + +VINS-Mono是香港科技大学沈劭劼团队于2018年开源的单目视觉惯导SLAM方案。是视觉与IMU融合中的经典之作，其定位精度可以媲美OKVIS，而且具有比OKVIS更加完善和鲁棒的初始化以及闭环检测过程。 + +VINS-Mono是基于优化和滑动窗口的VIO，使用IMU预积分构建紧耦合框架。并且具备自动初始化，在线外参标定，重定位，闭环检测，以及全局位姿图优化功能。由数据预处理、初始化、后端非线性优化、回环检测和全局位姿图优化五部分组成。 + +![VINS-Mono　框架图](https://github.com/DroidAITech/vslam-evaluation/raw/master/VINS/Installation%20documentation/pics/VINS-Mono%E3%80%80%E6%A1%86%E6%9E%B6%E5%9B%BE.png) + +VINS-Mono是目前开源SLAM里面效果非常不错的Visual和Inertial融合SLAM算法, 通过Sliding Windows优化进行视觉和IMU的紧耦合实现VIO, 此外还加了DBoW做Loop Closure, 算是一套完整的SLAM系统. 该算法的前端比较简单,就Harris角点加LK光流跟踪, Loop Closure也挺常规, 算法的厉害之处在于后端Visual Inertial融合优化部分, 也就是VIO部分, 这是VINS算法的核心。 + +但是，从实际运行效果来看： + +（1）vins纯旋转会飘，所以初始化必须平移加旋转，并且是快速地移动，当然移动是包括平移和旋转的； + +（2）初始化相对来讲有些麻烦，因为每次启动都要进行初始化； + +（3）运行过程中，视觉跟踪失败即一小段没有特征点或者移动太快的时候IMU会飘的很离谱； + +（4）尺度信息是依赖IMU的预积分值，而不是直接获取的，如此一来，就会产生误差，影响精度和鲁棒性； \ No newline at end of file -- Gitee From ae003d6cd9b13eb03d8cb831f3122251e0386502 Mon Sep 17 00:00:00 2001 From: anchuanxu Date: Fri, 8 Jul 2022 08:11:39 +0000 Subject: [PATCH 2/2] add LICENSE. --- LICENSE | 201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 201 insertions(+) create mode 100644 LICENSE diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..29f81d8 --- /dev/null +++ b/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. -- Gitee