(1)bark-ml

github开源bark-ml：https://github.com/bark-simulator/bark-ml/

bark官网：https://bark-simulator.github.io/

bark-ml指导手册：https://bark-simulator.github.io/tutorials/bark_ml_getting_started/

Youtube视频：https://www.youtube.com/user/fortissTV/search?query=BARK

相关论文：

• Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments (IV 2020)
• BARK: Open Behavior Benchmarking in Multi-Agent Environments (IROS 2020)
• Counterfactual Policy Evaluation for Decision-Making in Autonomous Driving (IROS 2020, PLC Workshop)

一、项目下载

# 新建一个Anaconda环境
conda create -n bark-ml python=3.7
# 如果想删除环境采用以下操作
# conda remove -n your_env_name(虚拟环境名称) --all
# 进入环境
conda activate bark-ml
pip install virtualenv==16.7.8
pip install bark-ml
git clone https://github.com/bark-simulator/bark-ml
cd bark-ml
1
2
3
4
5
6
7
8
9
10

二、配置环境

1、修改install.sh

which python  # 查看自己的python路径：/home/myx/anaconda3/envs/bark-ml/bin/python
sudo vim utils/install.sh  # 修改install.sh文件
1
2

将python3.7替换为刚刚得到的自己的python路径/home/myx/anaconda3/envs/bark-ml/bin/python

bash utils/install.sh
# 可能需要自己运行一下
# pip install torch==1.9.0+cpu torchvision -f https://download.pytorch.org/whl/torch_stable.html
# 进入虚拟环境
source utils/dev_into.sh
1
2
3
4
5

2、修改WORKSPACE

sudo gedit WORKSPACE
1

将原句

python_configure(name = “local_config_python”)

改为

python_configure(name = “local_config_python”, python_version = “3”)

三、测试

# source utils/dev_into.sh
# deactivate
bazel test //...
bazel test //bark_ml:unit_tests
bazel run //examples:continuous_env
bazel run //examples:blueprint_config
bazel run //examples:tfa
bazel run //examples:tfa_gnn -- --mode=train
bazel run //examples:tfa_counterfactual -- --mode=train
# 如果想训练
# bazel run //examples:tfa -- --mode=train
1
2
3
4
5
6
7
8
9
10
11

• 新建自己的参数文件夹 myx_params

/home/myx/Project/bark-ml/examples/myx_checkpoints/
1

/home/myx/Project/bark-ml/bark_ml/environments/blueprints/merging/merging.py
# 机器人行为和一大堆参数

/home/myx/anaconda3/envs/bark-ml/lib/python3.7/site-packages/tf_agents/drivers/dynamic_episode_driver.py
# 跑循环神经网络训练_loop_body_fn

/home/myx/Project/bark-ml/bark_ml/environments/blueprints/merging/merging.py
# evaluator 设置

/home/myx/Project/bark-ml/bark_ml/environments/single_agent_runtime.py
# reward, done, info = self._evaluator.Evaluate(

/home/myx/Project/bark-ml/bark_ml/evaluators/general_evaluator.py
  def Evaluate(self, observed_world, action):
    """Returns information about the current world state."""
    eval_results = observed_world.Evaluate()
    reward = 0.
    scheduleTerminate = False

    for _, eval_fn in self._bark_ml_eval_fns.items():
      t, r, i = eval_fn(observed_world, action, eval_results)
      eval_results = {**eval_results, **i} # merge info
      reward += r # accumulate reward
      if t: # if any of the t are True -> terminal
        scheduleTerminate = True

    return reward, scheduleTerminate, eval_results


/home/myx/anaconda3/envs/bark-ml/lib/python3.7/site-packages/tf_agents/agents/ppo/ppo_agent.py
# ppo 训练代码
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31