superset study day01 (本地启动superset项目)

什么是superset?

Apache Superset™ 是一个开源的现代数据探索和可视化平台。

Superset是一个现代化的数据探索和数据可视化平台。Superset 可以取代或增强许多团队的专有商业智能工具。Superset与各种数据源很好地集成。

Superset 提供：

• 用于快速构建图表的无代码界面
• 功能强大、基于 Web 的 SQL 编辑器，用于高级查询
• 轻量级语义层，用于快速定义自定义维度和指标
• 开箱即用，支持几乎任何 SQL 数据库或数据引擎
• 各种精美的可视化来展示您的数据，从简单的条形图到地理空间可视化
• 轻量级、可配置的缓存层，有助于减轻数据库负载
• 高度可扩展的安全角色和身份验证选项
• 用于编程定制的 API
• 从头开始设计的云原生架构，旨在实现规模化

superset文档

github文档:
https://github.com/apache/superset
官方操作文档
https://superset.apache.org/
文档某些步骤可能跑不通流程, 应该是版本迭代太快没有及时更新流程.

superset开发环境搭建

获取superset代码:

git clone https://github.com/apache/superset.git
1

superset后端环境

环境:
python 3.9.1
mysql 8.0
redis

1. 新建数据库

名称: superset_test
字符集: utf8mb4

2. 环境配置

# 安装 python3.9.1

# virtualenv创建虚拟环境
mkvirtualenv -p python  superset_test
workon superset_test

# conda创建虚拟环境
conda create -n superset_test python=3.9.1 -y
conda env list
conda activate superset_test

# 两种创建虚拟环境的任选一种

# install 
pip install apache-superset -i https://pypi.douban.com/simple
pip install flask_session==0.5.0 -i https://pypi.douban.com/simple
pip install pymysql==1.1.0 -i https://pypi.douban.com/simple
pip install requests==2.31.0 -i https://pypi.douban.com/simple
pip install Pillow==10.0.0 -i https://pypi.douban.com/simple
pwd 

pip install -e F:\zhondiao\github_superset\superset
# F:\zhondiao\github_superset\superset是当前项目路径
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

3. 修改py文件

config.py:

SECRET_KEY = "Y2Nrtdtt0nrKrMteiB2E/kgSr40OZW/z5ZiOqxGiJpsw/8RBV+lbGCqF"

SUPERSET_REDIS_HOST = "127.0.0.1"
SUPERSET_REDIS_PORT = "6379"
SUPERSET_REDIS_PASSWORD = ""
SUPERSET_REDIS_DB = 10


SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://root:123456@127.0.0.1:3306/superset_test?charset=utf8mb4'

WTF_CSRF_ENABLED = False
FAB_ADD_SECURITY_API = True

RATELIMIT_STORAGE_URI = (
    "redis://:"
    + SUPERSET_REDIS_PASSWORD +
    "@"
    + SUPERSET_REDIS_HOST +
    ":"
    + SUPERSET_REDIS_PORT +
    "/"
    + str(SUPERSET_REDIS_DB)
)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

superset\superset\models\ dynamic_plugins.py:

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
from flask_appbuilder import Model
from sqlalchemy import Column, Integer, Text, String

from superset.models.helpers import AuditMixinNullable


class DynamicPlugin(Model, AuditMixinNullable):
    id = Column(Integer, primary_key=True)
    name = Column(String(255), unique=True, nullable=False)
    # key corresponds to viz_type from static plugins
    key = Column(String(255), unique=True, nullable=False)
    bundle_url = Column(String(255), unique=True, nullable=False)

    def __repr__(self) -> str:
        return str(self.name)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

superset\superset\models\ sql_lab.py

# ...

    latest_query_id = Column(
        String(11), ForeignKey("query.client_id", ondelete="SET NULL")
    )
#    latest_query_id = Column(
#        Integer, ForeignKey("query.client_id", ondelete="SET NULL")
#    )
    
# ...
1
2
3
4
5
6
7
8
9
10

superset\superset\tables\ models.py

    catalog = sa.Column(sa.String(50))
    schema = sa.Column(sa.String(50))
    name = sa.Column(sa.String(50))
    # catalog = sa.Column(sa.Text)
    # schema = sa.Column(sa.Text)
    # name = sa.Column(sa.Text)
1
2
3
4
5
6
7

app.py

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

import logging
import os
from typing import Optional

from flask import Flask

from superset.initialization import SupersetAppInitializer

logger = logging.getLogger(__name__)


def create_app(superset_config_module: Optional[str] = None) -> Flask:
    app = SupersetApp(__name__)

    try:
        # Allow user to override our config completely
        config_module = superset_config_module or os.environ.get(
            "SUPERSET_CONFIG", "superset.config"
        )
        app.config.from_object(config_module)

        app_initializer = app.config.get("APP_INITIALIZER", SupersetAppInitializer)(app)
        app_initializer.init_app()

        return app

    # Make sure that bootstrap errors ALWAYS get logged
    except Exception as ex:
        logger.exception("Failed to create app")
        raise ex


class SupersetApp(Flask):
    pass


# 本地测试
if __name__ == '__main__':
    superset_app = create_app()
    # print(superset_app.url_map)
    superset_app.run(
        host="0.0.0.0", port=5000, debug=True,
    )

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

superset\superset\migrations\ script.py.mako

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
"""${message}

Revision ID: ${up_revision}
Revises: ${down_revision}
Create Date: ${create_date}

"""

# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}

from alembic import op
import sqlalchemy as sa
import sqlalchemy_utils
${imports if imports else ""}

def upgrade():
    ${upgrades if upgrades else "pass"}


def downgrade():
    ${downgrades if downgrades else "pass"}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

清空superset\superset\migrations\versions文件下所有的py文件

4. 迁移数据库

superset db upgrade

superset db migrate

superset db upgrade

superset fab create-admin

superset init 
1
2
3
4
5
6
7
8
9

创建admin用户:

5. 启动项目

本次测试先运行app.py

Python app.py

在此之前,要打包前端代码,才能进去系统里

superset 前端代码打包

准备:
nodejs

git pull
cd superset-frontend
npm install  --registry=https://registry.npm.taobao.org    

npm run build

1
2
3
4
5
6

搭建完成,效果页面