ag真人试玩

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

身边有很多运维工程师,做了几年的运维自动化,但依然不能确定选择哪个工自动化工作?还有,怎样更优雅的实施运维自动化和避免事实当中的坑?
马哥教育大咖讲堂-175期特别邀请了一线资深工程师张强为我们带来了一场《saltstack在自动化运维实践》的分享,尝试带着他对工作问题的思考以及部署自动化工具的经验给大家一些帮助和启发。
————————————
作者介绍:
张强,马哥教育Linux大咖讲堂金牌讲师,现就职于伙伴智慧运维工程师,负责主线业务平台,3年Linux一线经验,擅长shell脚本、自动化_发布、web应用等,现在关注自动化运维、分布式数据库,虚拟化技术。


自动化工具比较

puppetyexushisikuangongjuzhongzuishenrurenxinde。jiukeyongcaozuo、mokuaiheyonghujiemianeryan,tashizuiquanmiande。puppetchengxianleshujuzhongxinxiediaodequanmao,jihuhangaimeiyigeyunxingxitong,weigedacaozuoxitongtigongleshenrudegongju。chushishezhibijiaojiandan,zhixuyaozaixuyaojiayiguanlidemeigexitongshanganzhuangzhufuwuqihekehuduandailiruanjian。minglingxingjiekou(cli)jiandanzhiguan,yunxutongguopuppetminglingxiazaiheanzhuangmokuai。ranhou,xuyaoduipeizhiwenjianjinxinggenggai,haorangmokuaishihesuoxuderenwu;yingjiedaozhilingdekehuduanyuzhufuwuqilianxishi,huigenggaipeizhiwenjian,huozhekehuduantongguolijichufagenggaipeizhiwenjiandetuisong(push)laijinxinggenggai。

ag真人试玩ansibleguanzhudezhongdianshiliqiujingjianhekuaisu,erqiebuxuyaozaijiedianshanganzhuangdailiruanjian。yinci,ansibletongguosshzhixingsuoyougongneng。xuyaoguanlidejiedianbeitianjiadaoansiblepeizhihuanjing,sshshouquanmiyaobeifujiadaomeigejiedianshang,zheiyuyunxingansibledeyonghuyouguan。yidanwanchenglezheibu,ansiblezhufuwuqikeyitongguosshyujiedianjinxingtongxin,zhixingsuoyoubiyaoderenwu。ansiblekeyishiyongparamiko(jiyussh2xieyidepythonshixian)huobiaozhunsshyongyutongxin,buguohaiyouyizhongjiasumoshi,yunxugengkuaisu、gengdaguimodetongxin。

ag真人试玩saltleisiansible,yinweitayeshijiyuclidegongju,caiyongletuisongfangfashixiankehuduantongxin。takeyitongguogithuotongguochengxubaoguanlixitonganzhuangdaozhufuwuqihekehuduanshang。kehuduanhuixiangzhufuwuqitichuqingqiu,qingqiuzaizhufuwuqishangdedaojieshouhou,jiukeyikongzhigaikehuduanle。saltkeyitongguoputongdesshyukehuduanjinxingtongxin,danruguoshiyongmingweiminiondekehuduandailiruanjian,keyidadazengqiangkekuozhanxing。ciwai,salthanyouyigeyibuwenjianfuwuqi,keyiweikehuduanjiakuaiwenjianfuwusudu,zheiwanquanshisaltzhuzhonggaokuozhanxingdeyigetixian。yuansibleyiyang,nikeyizhijietongguocli,xiangkehuduanfachumingling,biruqidongfuwuhuoanzhuangchengxubao;niyekeyishiyongmingweistatedeyamlpeizhiwenjian,chulibijiaofuzaderenwu。haiyou“pillar”,zheixieshifangzaijizhongdifangdeshujuji,yamlpeizhiwenjiankeyizaiyunxingqijianfangwentamen。

zongjie:gerenguandianpuppetzuidaquedianjiushimorenqingkuangxiaagentmeige30fenzhongxiangmastertongbuzhuangtai,masterzhudongtuisonggongnengbijiaoboruo(2.7banben),ansiblejiyusshfuwuzhixing,ruguofuwuqiguoduobujianyishiyong,tashishiyonglunxundefangshi。saltjiyuxiaoxiduilie。xingnengxiangdanghao,shihedaliangshengchanhuanjing。

SaltStack简介与特性

saltstack shiyizhongjiyu c/s jiagoudefuwuqijichujiagoujizhonghuaguanlipingtai,guanliduanchengwei master,kehuduanchengwei minion。saltstack jubeipeizhiguanli、yuanchengzhixing、jiankongdenggongneng,yibankeyilijieweishijianhuabande puppet hejiaqiangbande func。saltstack benshenshijiyu python yuyankaifashixian,jieheleqingliangjidexiaoxiduilieruanjian zeromq yu python disanfangmokuai(pyzmq、pycrypto、pyjinjia2、python-msgpack he pyyaml deng)goujian。

tongguobushu saltstack huanjing,yunweirenyuankeyizaichengqianshangwantaifuwuqishangzuodaopiliangzhixingmingling,genjubutongdeyewutexingjinxingpeizhijizhonghuaguanli、fenfawenjian、caijixitongshujujiruanjianbaodeanzhuangyuguanlideng。

SaltStack 具有以下特性:

1、bushujiandan、fangbian;

2、zhichidabufenunix/linuxjiwindowshuanjing;

3、zhucongjizhonghuaguanli;

4、peizhijiandan、gongnengqiangda、kuozhanxingqiang;

5、zhukongduan(master)hebeikongduan(minion)jiyuzhengshurenzheng,anquankekao。

6、zhichiapijizidingyimokuai,ketongguopythonqingsongkuozhan。

SaltStack 的工作原理

ag真人试玩saltstack caiyong c/s jiegoulaiduiyunhuanjingneidefuwuqicaozuoguanlijipeizhiguanli。weilegenghaodelijietadegongzuofangshijiguanlimoxing,jiangtongguotuxingfangshiduiqiyuanlijinxingchanshu。

saltstack kehuduan(minion)zaiqidongshi,huizidongshengchengyitaomiyao,baohansiyaohegongyao。zhihoujianggongyaofasongjifuwuqiduan,fuwuqiduanyanzhengbingjieshougongyao,yicilaijianlikekaoqiejiamidetongxinlianjie。tongshitongguoxiaoxiduilie zeromq zaikehuduanyufuwuduanzhijianjianlixiaoxifabulianjie。jutitongxinyuanlitu,rutu 1 suoshi,minglingzhixingrutu 2 suoshi:

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

zhuanyeshuyushuoming:

minion shi saltstack xuyaoguanlidekehuduananzhuangzujian,huizhudongqulianjie master duan,bingcong master duandedaoziyuanzhuangtaixinxi,tongbuziyuanguanlixinxi。

ag真人试玩master zuoweikongzhizhongxinyunxingzaizhujifuwuqishang,fuze salt minglingyunxingheziyuanzhuangtaideguanli。

zeromq shiyikuankaiyuandexiaoxiduilieruanjian,yongyuzai minion duanyu master duanjianlixitongtongxinqiaoliang。

daemon shiyunxingyumeiyigechengyuanneideshouhujincheng,chengdanzhefabuxiaoxijitongxinduankoujiantingdegongneng。

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

yuanlitushuoming:

minion shi saltstack xuyaoguanlidekehuduananzhuangzujian,huizhudongqulianjie master duan,bingcong master duandedaoziyuanzhuangtaixinxi,tongbuziyuanguanlixinxi。

master zuoweikongzhizhongxinyunxingzaizhujifuwuqishang,fuze salt minglingyunxingheziyuanzhuangtaideguanli。

ag真人试玩master shangzhixingmoutiaozhilingtongguoduiliexiafadaogege minions quzhixing,bingfanhuijieguo。

SaltStack 的架构设计

ag真人试玩weilerangdajiagenghaodelijie saltstack jizhonghuaguanlifangmiandeyoushi,yinci,genjuxiangmudeshijiqingkuanghuizhilebushujiagoutu,bingzaiwenzhongduijiagoutujinxinglexiangxishuoming。rutu 3 suoshi:

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

shuoming:

saltstack desuoyoubeiguanlikehuduanjiedian(rutu 3 suoshi db he web),doushitongguomiyaojinxingjiamitongxin,shiyongduankouwei 4506。kehuduanyufuwuqiduandeneirongchuanshu,shitongguoxiaoxiduiliewancheng,shiyongduankouwei 4505。master keyifasongrenhezhilingrang minion zhixing,salt youhenduokezhixingmokuai,birushuo cmd mokuai,zaianzhuang minion deshihouyijingzidaile,tamentongchangweiyunide python kuzhong,locate salt | grep /usr/ keyikandao salt zidaidesuoyoudongxi。

ag真人试玩weilegenghaodelijiejiagouyongyi,yixiajiangzhanshizhuyaodeminglingfabuguocheng:

saltstack de master yu minion zhijiantongguo zeromq jinxingxiaoxichuandi,shiyongle zeromq defabudingyuemoshi,lianjiefangshibaokuo tcp he ipc。

ag真人试玩salt mingling,jiang cmd.run ls minglingcong salt.client.localclient.cmd_cli fabudao master,huoquyige jodid,genju jobid huoquminglingzhixingjieguo。

master jieshoudaominglinghou,jiangyaozhixingdeminglingfasongjikehuduan minion。

minion congxiaoxizongxianshangjieshoudaoyaochulidemingling,jiaoji minion._handle_aes chuli。

minion._handle_aes faqiyigebendixianchengdiaoyong cmdmod zhixing ls mingling。xianchengzhixingwan ls hou,diaoyong minion._return_pub fangfa,jiangzhixingjieguotongguoxiaoxizongxianfanhuiji master。

master jieshoudaokehuduanfanhuidejieguo,diaoyong master.handle_aes fangfajiangjieguoxiedewenjianzhong。

ag真人试玩salt.client.localclient.cmd_cli tongguolunxunhuoqu job zhixingjieguo,jiangjieguoshuchudaozhongduan。

SaltStack 的安装与配置

dui saltstack youleyigechubudelejiezhihou,tongguoshijianlicaozuojinyibulejie saltstack。

yi、anzhuangsalt

saltxuyaoepelyuanzhichi,suoyouanzhuangqianxuyaoxiananzhuangepelyuanbao。

ag真人试玩1、salt-master

ag真人试玩# yum -y install salt-master

2、salt-minion

ag真人试玩# yum -y install salt-minion

er、peizhisalt

1、master(/etc/salt/master)

# saltyunxingdeyonghu,yingxiangdaosaltdezhixingquanxian

user: root

#saltdeyunxingxiancheng,kaidexianchengyueduoyibanchulidesuduyuekuai,danyibanbuyaochaoguocpudegeshu

ag真人试玩worker_threads: 10

# masterdeguanliduankou

publish_port : 4505

# mastergenminiondetongxunduankou,yongyuwenjianfuwu,renzheng,jieshoufanhuijieguodeng

ret_port : 4506

# ruguozheigemasteryunxingdesalt-syndiclianjiedaoleyigegenggaocengjidemaster,neimezheigecanshuxuyaopeizhichenglianjiedaodezheigegaocengjimasterdejiantingduankou

ag真人试玩syndic_master_port : 4506

# zhidingpidwenjianweizhi

ag真人试玩pidfile: /var/run/salt-master.pid

ag真人试玩# saltstack keyikongzhidewenjianxitongdekaishiweizhi

root_dir: /

# rizhiwenjiandizhi

log_file: /var/log/salt_master.log

# fenzushezhi

nodegroups:

group_all: ‘*’

ag真人试玩# salt statezhixingshihoudegenmulu

file_roots:

base:

ag真人试玩– /etc/salt/

# shezhipillar degenmulu

pillar_roots:

base:

ag真人试玩– /etc/pillar

ag真人试玩2、peizhiminion(/etc/salt/minion)

ag真人试玩master: mail  #zheikuaidemailzhideshizai/etc/hostswenjianzhongsuodingyidezhujiming

id: node1

3、qidongsalt

ag真人试玩service salt-master start

service salt-minion start

# saltstack shishiyongpython2deyuyanbianxie,duipython3dejianrongxingbuhao,qingshiyongpython2dehuanjing

4、renzhengminglingjieshao

salt-key #zhengshuguanli

ag真人试玩# salt-key –l       #chakansuoyouminion-key

# salt-key –a      #jieshoumougeminion-key

# salt-key –a      #jieshousuoyouminion-key

# salt-key –d       #shanchumougeminion-key

ag真人试玩# salt-key –d       #shanchusuoyouminion-key

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

5、saltminglingjieshao

ag真人试玩minglinggeshi:salt [options] [arguments]

li:salt \* cmd.run ‘uptime’

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

SaltStack minion匹配方式

1、 glob(saltmorendetargetleixing,shiyongshelldetongpeifulaizhidingyigehuoduogeminion id)

ag真人试玩# salt \* test.ping huo salt ‘*’ test.ping

2、pcrejianrongzhengzebiaodashi

# salt –e ‘^[m|m]in.[e|o|u]n$’ test.ping

3、subnet(tongguozhidingyigeipv4dizhihuoyigecidrdeipv4ziwang)

# salt –s 192.168.0.42 test.ping

# salt –s 192.168.0.0/16 test.ping

4、grains(saltkeyitongguocaozuoxitong、cpujiagoujizidingyixinxidengjiqitezhengjinxingtarget minion)

# salt –g ‘os:ubuntu’ test.ping

# salt –g ‘os_family:debian’ test.ping

ag真人试玩5、pillar(saltzhichitongguopillarshujujinxingpipei)

ag真人试玩# salt –i ‘my_val:my_val’ test.ping

ag真人试玩6、hunhe(compound)

ag真人试玩# salt –c ‘web* or g@os:arch’ test.ping

ag真人试玩7、jiedianzu(nodegroup)

ag真人试玩jiedianzuxuyaoshixiandingyi,peizhifangfaruxia:

ag真人试玩# vim /etc/salt/master

nodegroups:

node: ‘l@node1,node2’

ag真人试玩# salt -n node test.ping

SaltStack常用模块

ag真人试玩1、statusmokuai(chakanxitongxinxi)

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

ag真人试玩# salt “*” status.diskstats    #chakancipanxinxi

ag真人试玩# salt “*” status.meminfo      #chakanneicunxinxi

ag真人试玩# salt “*” status.w            #wminglingfanhuixinxi

2、chakansuoyoumoduleliebiao

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

ag真人试玩3、chakanzhidingmoduledesuoyoufunctionfangfa

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

4、chakanzhidingmoduleyongfa

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

5、jutimokuaideshiyong(lizi)

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

tongshidaozhidingjiqichakan

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

cmd.runmokuaideshiyong

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

Grains

Static bits of information that a minion collects about the system when the minion first starts.

the grains interface is made available to salt modules and components so that the right salt minion commands are automatically available on the right systems.

以上是官方的解释,大致意思是说grains是minion第一次启动的时候采集的静态数据,可以用在salt的模块和其他组件中。例如,当os_family的Grain数据为Centos时,则会使用yum工具组件来进行软件包管理。Grains会在Minion进程启动时加载,并缓存在内存中。ag真人试玩这样salt-minion进程就无须每次操作都重新检索系统来获取Grain,极大的提高了Minion的性能。

ag真人试玩1、womenzheilijiandanzuoyigeshuchuceshi,keyikandaominionjiediandeyixiexinxiruxia:

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

chakanjutimeiyixiangxinxi

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

2、yingyongchangjing:

grainsdetexing–meiciqidonghuibao、jingtaijuedingletameiyoupillarlinghuo,yaozhidaopillarshisuishikebiande,zhiyaozaimasterduanxiugaileneiyibandouhuilikeshengxiaode。suoyigrainsgengshihezuoyixiejingtaideshuxingzhidecaiji,lirushebeidejiaose(role),cipangeshu(disk_num),caozuoxitongbanbendengzhurucileifeichanggudingdeshuxing。jiandanzongjieqilaigrainsdeyongturuxia:

ag真人试玩(1),grainskeyizaistatexitongyingyongzhong,yonghupeizhiguanlimokuai。

ag真人试玩(2),grainskeyizaitargetzhongshiyong,yonglaipipeiminion,biruos,yong-g。

ag真人试玩(3),grainskeyiyongyuxinxichaxun,grainsbaocunzheshoujidaodekehuduandexinxi。

neimewomenjiukeyidedaoyigedazhidepanduan,ruguonixiangdingyideshuxingzhishijingchangbianhuade,neiqingcaiyongpillar,ruguoshihenguding、buyibiandeneiqingyonggrains。

ag真人试玩3、grainsyouxianji

ag真人试玩grainskeyibaochizaiminionduan、tongguomasterduanxiafadengduogefangshilaifenfa。danbutongdefangfayoubutongdeyouxianjide(youdidaogao):

ag真人试玩(1). /etc/salt/grains

(2) /etc/salt/minion

ag真人试玩(3)./srv/salt/_grains/  masterduan_grainsmuluxia

youxianjishunxuyiciweicunzaizaiminionduan/etc/salt/minionpeizhiwenjianzhongdetongminggrainshuifugai/etc/salt/grainswenjianzhongdezhi,ertongguomasterduan_grainsmuluxiagrainswenjianxiafadezhikeyihuifugaiminionduandesuoyoutongmingzhi。bijiaoaokou,zongzhijide,tongguomasterxiafadegrainsyouxianjishizuigaodeke,/etc/salt/minioncizhi,/etc/salt/grainszuidi(core grainsbudadong,jiubutaolunle,zheigebi/etc/salt/grainshaidi)。

4、grainsdexiafa

grainsdexiafadazhikeyifenweilianggesilu:

(1)zidingyide(_grains)keyitongguostate.highstate、saltutil.sync_grains、saltutil.sync_all dengfangfapiliangxiafa,qiejisuoyouzai_grainsmuluxiadesuoyouzidingyigrainszhidouhuixiafadaominion,zheishixuedejiaoxun。

ag真人试玩(2)gudingcunfangzaiminionduanpeizhiwenjianzhong,rugrains、minionwenjianzhong,keyitongguofile managerdefangfaqupiliangxiafa/etc/salt/grainsdengpeizhiwenjianshixiangrainsdepiliangxiafa,dangranleyetongguobiedefangshibazheigewenjianpiliangxiafaxiaqu,doushiokde。

duibi:

(1)tongguostate.highstate xiafadegrainshaochushiwuxuzhongqiminionjikeshengxiao,dantongguoxiafa/etc/salt/grainswenjianxiafadegrainszhizebixuzhongqiminionduanfuwucaikeyishengxiao。

ag真人试玩(2)zidingyide_grainsmeicizaihighstatediaoyongdeshihoujiuhuizidongxiafa、shuaxin,er/etc/salt/grainswenjiandezebuhui。

Pillar

zaidaduoshuchangjingzhong,pillardebiaoxianxingweihegrainyizhi,danyougehendadequbieshi:pillarzaimastershangjinxingdingyi,cunzaiyuyigejizhonghuadelujing。pillarshujushiyutedingminionguanliande,yejiushishuomeiyigeminiondouzhinengkandaozijideshuju,suoyipillarkeyiyonglaichuandiminganshuju(zaisaltdeshejizhong,pillarshiyongdulidejiamisession,yeshiweilebaozhengminganshujudeanquanxing)。

pillarkeyiyongzaineixiedifang:

1、minganshuju

ag真人试玩lirussh key,jiamizhengshudeng,youyupillarshiyongdulidejiamisession,keyiquebaozheixieminganshujububeiqitaminionkandao。

2、bianliang

keyizaipillarzhongchulipingtaichayixing,biruzhenduibutongdecaozuoxitongshezhiruanjianbaodemingzi,ranhouzaistatezhongyinyong。

3、qitarenheshuju

ag真人试玩keyizaipillarzhongtianjiarenhexuyaoyongdaodeshuju。birudingyiyonghuheuiddeduiyingguanxi,mniondejiaosedeng。

4、yongzaitargettingzhong

ag真人试玩pillarkeyiyonglaixuanzeminion,shiyong-ixuanxiang。

dingyipillar:

ag真人试玩masterpeizhiwenjianzhongdingyi:

ag真人试玩morenqingkuangxia,masterpeizhiwenjianzhongdesuoyoushujudoutianjiadaopillarzhong,qieduisuoyouminionkeyong。ruguoyaojinyongzheiyimorenzhi,keyizaimasterpeizhiwenjianzhongtianjiaruxiashuju,zhongqifuwuhoushengxiao:

pillar_opts: false

shiyongslswenjiandingyipillar

ag真人试玩pillarshiyongyustatexiangsideslswenjian。pillarwenjianfangzaimasterpeizhiwenjianzhongpillar_rootsdingyidemuluxia。shiliruxia:

ag真人试玩pillar_roots:

base:

ag真人试玩– /srv/pillar

xiamianzheiduandaimadingyilebasehuanjingxiadepillarwenjianbaocunzai/srv/pillar/muluxia。yustatexiangsi,pillaryeyoutop file,yeshiyongxiangtongdepipeifangshijiangshujuyingyongdaominionshang。shiliruxia:

ag真人试玩# cat /srv/pillar/top.sls:

base:

ag真人试玩‘*’:

– packages

ag真人试玩# cat /srv/pillar/packages.sls:

{% if grains[‘os’] == ‘redhat’ %}

apache: httpd

git: git

{% elif grains[‘os’] == ‘debian’ %}

ag真人试玩apache: apache2

ag真人试玩git: git-core

{% endif %}

basehuanjingzhongsuoyoudeminiondoujuyoupackageszhongdingyideshuju。pillarcaiyongyufile serverxiangtongdewenjianyingshefangshi,zaibenlizhong,packagesyingshedaowenjian/srv/pillar/packages.sls。zhuyikeyyuvalueyaoyongmaohaojiakonggefenge,meiyoukonggedehuajiangjiexishibai。

ag真人试玩ruhezhidaominionyongyouneixiepillarshuju?

zaimastershangxiugaipillarwenjianhou,xuyaoyongyixiaminglingshuaxinminionshangdeshuju:

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

shiyongpillarhuoquzidingyishuju:

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

State

ag真人试玩jianshu:sls(daibiaosalt statewenjian)shisalt statexitongdehexin。slsmiaoshulexitongdemubiaozhuangtai,yougeshijiandandeshujugoucheng。zheijingchangbeichengzuopeizhiguanli

ag真人试玩top.slsshipeizhiguanliderukouwenjian,yiqiedoushicongzheilikaishi,zaimaster zhujishang,morencunfangzai/srv/salt/mulu.

top.sls morencong base biaoqiankaishijiexizhixing,xiayijishicaozuodemubiao,keyitongguozhengze,grainmokuai,huofenzuming,laijinxingpipei,zaixiayijishiyaozhixingdestatewenjian,bubaohuankuozhanming。

chuangjian/srv/salt/top.sls

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

stateshizhan

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

老司机独爱Linux自动化运维神器saltstack,看他60分钟的真言

————广告时间————

《马哥Linux云计算及架构师》课程,由知名Linux布道师马哥创立,经历了8年的发展,联合阿里巴巴、唯品会、大众点评、腾讯、陆金所等大型互联网一线公司的马哥课程团队的工程师进行深度定制开发,课程采用 Centos7.2系统教学,加入了大量实战案例,授课案例均来自于一线的技术案例。

开课时间:11月06号

相关新闻