本项目简介：
一般分为硬件端和软件端，硬件端包括：esp32连接面包板，画画pcb焊接调试电路图pcb电路，最后用三维设计外壳。软件端包括:使用python编写的后端服务器，页面显示的客户端。(本项目应使用相关乐理知识)
我们项目的总流程图
文字描述：esp32硬件先连接WiFi，蓝灯亮了，说明连接成功，不亮就用蜂鸣器发声，亮了，esp按下32个按钮，按下相应的按钮udp通信发送到服务端，服务端收到udp信息 可以利用pygame也可以通过服务端和前端发声websocket连接，将信息发送到前端，前端收到信息，实现相应的键变色，记录简谱。
也可以按下前端的琴键，让服务端pygame声音，变色，记录简谱。
扩展部分:1。作弊模式:硬件按下按钮，无论硬件按哪个按钮，前端已经规定的音谱都会发出。
软件端的效果如下：
一 后端（thonny）
1.使esp32发出不同的声音
2.esp32发声(奏乐)
2.录制esp32发音，判断按键的长度
3.利用pygame发声
4.用udp连接使服务器与硬件建立联系，硬件按键服务端接收udp信息，发出对应的音调
5.esp32的WiFi连接
二 前端（vscode）
1.在页面上绘制UI琴键
2.构建电子琴结构
3.建立服务端和前端websocket连接
4.硬件发送udp通过向服务端收到信息websocket将连接发送到前端
以下是扩展部分：
1.打开作弊模式
2.切换上一首和下一首，清除页面前的音符，清空页面上的所有音符
3.播放音乐
硬件端的效果如下：
1.esp32连接在面包板上
2.画pcb电路图
3.pcb焊接电路板
4.设计和打印三维模型
三维打印、前端页面优化和乐器多重奏：
1.前端优化
二、三维设计，pcb锂电池的焊接和供电
3.硬件调试(按定义音调演奏)
坚持就是胜利
------------------------------------------------------------------------------
做之前要了解基础乐理知识:音乐基础乐理知识 | 乐理知识 - 知乎 (zhihu.com)
在发音之前，我们将了解蜂鸣器的相关知识：蜂鸣器的原理 - 单片机教程 - C语言网 (dotcpp.com)
ESP32 MicroPython上手指南 — MicroPython 1.14 文档 (01studio.org)esp32官网：ESP32 MicroPython上手指南 — MicroPython 1.14 文档 (01studio.org)
了解面包板:(4条消息) 面包板使用简介_countofdane的博客-CSDN博客_面包板的详细使用方法
使用的硬件有esp32 剥线钳 导线 蜂鸣器(下图为esp32）
"一切都准备好了，只欠东风 --------------------------------"
首先，我们需要先将军esp32在面包板上完成连接
完成连接后，我们可以实现功能
①按下面包板上的琴键，发出不同的音调，点亮蓝灯
代码如下：
from machine import TouchPad, Pin,PWM from time import sleep
LED = Pin(2,Pin.OUT)
pwm0 = PWM(Pin(23)) freq_val=5 duty_val=0 pwm0.freq(freq_val) pwm0.duty(duty_val)
PINn = #元组#
TPx = [TouchPad(Pin(i)) for i in PINn]#TPx 触摸管脚的所有定义
Tone2 = (#管脚对应演奏声音的频率 523, 586, 658, 697, 783, 879, 987, 1045, )
def dzq(): global freq_val,duty_val 电子琴程序 for ton,tp in zip(Tone2,TPx): 所有触摸按钮遍历 if tp.read()<200: freq_val=ton duty_val = 1024//2 LED.on() break else: #break，就执行这个else' freq_val = 10 duty_val = 0 LED.off() freq1 = pwm0.freq()#读取当前频率 # pwm0.freq(freq_val) # sleep(0.01) # pwm0.duty(duty_val) if freq1 != freq_val: 只有当频率被修改时，才能重新配置频率和占空比 pwm0.freq(freq_val) sleep(0.01)#需要一定时间的延时，否则的话，同时修改评论和占空比会导致占空比修改失败 pwm0.duty(duty_val) # while pwm0.duty() != duty_val: # pwm0.duty(duty_val) # print(pwm0.freq(),pwm0.duty()) while True: dzq() sleep(0.01)
------------------------------------------------------------------------
②实现录制播放并且记录摁下时间长短：
import time #引入时间类 from machine import TouchPad, Pin, PWM #引入触摸管脚 import threading
switch = TouchPad(Pin(32)) switch_mode = 0
PINn = (33,27,13,12,14,15,4)#元组 TPx = [TouchPad(Pin(i)) for i in PINn]#TPx 所有的触摸管脚定义
LED = Pin(2, Pin.OUT)
pwm0 = PWM(Pin(26)) time.sleep(0.5) pwm0.duty(0) time.sleep(0.01) pwm0.freq(1)
Tone = (#对应管脚要演奏声音的频率 523, 586, 658, 697, 783, 879, 987, )
my_rhythm = [] #创建节拍空数组 my_tones = [] #创建音调空数组
release=0 #松开状态 press=1 #按下状态 key_state = [release]*8 #按键状态缓存变量 ALLkey_state = release start = time.ticks_ms() #记录开始的时间戳 end = 0 #记录结束的时间戳 time_diff = 0 #开始和结束的时间差
def Record_time(): global start,end,time_diff end = time.ticks_ms() #记录这一次按键变化结束的时间戳 time_diff = time.ticks_diff(end, start) #根据这一次按键变化的开始和结束时间戳，计算出按键变化的时间差 my_rhythm.append(time_diff/1000) #将时间差存入数组末尾 # print(my_rhythm) #打印数组 #----------------------------------------------------- start = time.ticks_ms() #记录下一次按键变化开始的时间戳
def playtone(frequency): pwm0.duty(512) time.sleep(0.01) pwm0.freq(frequency)
def bequiet(): pwm0.duty(0) time.sleep(0.01) pwm0.freq(1)
def playsong(): global switch_mode for i in range(len(my_rhythm)): if (my_tones[i] == 0 ): bequiet() else: playtone(my_tones[i]) time.sleep(my_rhythm[i]) bequiet() switch_mode=0 print('music_OK')
my_music = threading.Thread(target=playsong)
def switchkey(): global switch_mode,start while True: if switch.read()<200: if switch_mode==0: my_rhythm.clear() my_tones.clear() switch_mode=1 start = time.ticks_ms() #记录开始的时间戳 my_tones.append(0) #第一个项为空拍，与节拍数组格式对应 LED.on() elif switch_mode==1: Record_time() print(my_tones) print(my_rhythm) switch_mode=2 LED.off() my_music.start() while switch.read()<200: time.sleep(0.01) sw_key = threading.Thread(target=switchkey) sw_key.start()
while True: for i in range(7): #循环7次，读取7个按键状态 if TPx[i].read()<200: #读取按键电容值，判断按键是否按下 if key_state[i] != press: #如果按键状态缓存变量 非 此按键键值，表示按键发生了改变 key_state[i] = press #将按键状态缓存变量 置为 此按键键值 if switch_mode==1: Record_time() #记录按键改变的时间差 my_tones.append(Tone[i]) #记录当前按键的音调 playtone(Tone[i]) #发出对应频率的音调 ALLkey_state = press #表示有按键按下 else: key_state[i] = release #将当前按键状态置为松开 if press not in key_state: if ALLkey_state != release : #全部松开状态下 判断之前是否有按键按下 ALLkey_state = release #将按键状态置为全部松开状态 if switch_mode==1: Record_time() #记录按键改变的时间差 my_tones.append(0) #记录当前空拍音调 bequiet() #不发声 time.sleep(0.1) #延时
③利用pygame发声 a~z摁下都可以发声，记录并打印摁下时间：
import pygame.midi import pygame import time
# 初始化设置 volume = 127 # 音量 0-127 pygame.init() # 初始化PYgame windowSurface = pygame.display.set_mode((800, 600)) # 建立窗口 device = 0 # device number in win10 laptop instrument = 0 # 乐器 http://www.ccarh.org/courses/253/handout/gminstruments/ # initize Pygame MIDI ---------------------------------------------------------- pygame.midi.init() # PYGAMEMIDI库的初始化 # 初始化设置结束
screen = pygame.display.set_mode((400,400))
# 设置窗口的标题，即游戏名称 pygame.display.set_caption('pygame 钢琴')
# 引入字体类型 f = pygame.font.Font('C:/Windows/Fonts/simhei.ttf',135) # 生成文本信息，第一个参数文本内容；第二个参数，字体是平滑； # 第三个参数，RGB模式的字体颜色；第四个参数，RGB模式字体背景颜色； text = f.render("Zhang",True,(255,244,255),(31,56,99)) #获得显示对象的rect区域坐标 textRect =text.get_rect() # 设置显示对象居中 textRect.center = (200,200) # 将准备好的文本信息，绘制到主屏幕 Screen 上。 screen.blit(text,textRect)
# 固定代码段，实现点击"X"号退出界面的功能，几乎所有的pygame都会使用该段代码 Tone = { # 音调字典，不全，需要大家完善。从C1-C5都完善起来 'A0':21,'A#0':22,'B0':23, 'C1':24,'C#1':25,'D1':26,'D#1':27,'E1':28,'F1':29,'F#1':30,'G1':31,'G#1':32,'A1':33,'A#1':34,'B1':35, 'C2':36,'C#2':37,'D2':38,'D#2':39,'E2':40,'F2':41,'F#2':42,'G2':43,'G#2':44,'A2':45,'A#2':46,'B2':47, 'C3':48,'C#3':49,'D3':50,'D#3':51,'E3':52,'F3':53,'F#3':54,'G3':55,'G#3':56,'A3':57,'A#3':58,'B3':59, 'C4':60,'C#4':61,'D4':62,'D#4':63,'E4':64,'F4':65,'F#4':66,'G4':67,'G#4':68,'A4':69,'A#4':70,'B4':71, 'C5':72,'C#5':73,'D5':74,'D#5':75,'E5':76,'F5':77,'F#5':78,'G5':79,'G#5':80,'A5':81,'A#5':82,'B5':83, 'C6':84,'C#6':85,'D6':86,'D#6':87,'E6':88,'F6':89,'F#6':90,'G6':91,'G#6':92,'A6':93,'A#6':94,'B6':95, 'C7':96,'C#7':97,'D7':98,'D#7':99,'E7':100,'F7':101,'F#7':102,'G7':103,'G#7':104,'A7':105,'A#7':106,'B7':107, 'C8':108, } # set the output device -------------------------------------------------------- player = pygame.midi.Output(device) # 定义了一个输出音轨
# set the instrument ----------------------------------------------------------- player.set_instrument(instrument) # 设置乐器音色
key_value = ('a','s','d','f','g','h','j','q','w','e','r','t','y','u','z','x','c','v','b','n','m','1','2','3','4','5','6','7','i','o','p','k','l',) key_tone = { 'a':"C2",'s':"D2",'d':"E2",'f':"F2",'g':"G2",'h':"A2",'j':"B2", 'q':"C3",'w':"D3",'e':"E3",'r':"F3",'t':"G3",'y':"A3",'u':"B3", 'z':"C1",'x':"D1",'c':"E1",'v':"F1",'b':"G1",'n':"A1",'m':"B1", '1':"C4",'2':"D4",'3':"E4",'4':"F4",'5':"G4",'6':"A4",'7':"B4", 'i':'C5','o':'D5','p':'E5','k':'F5','l':'G5', }
while True: for event in pygame.event.get(): # 检测事件 if event.type == pygame.QUIT: exit() if event.type == pygame.KEYDOWN: for i in range(33): if event.key == pygame.__dict__[ 'K_'+key_value[i] ]: print('正在发第'+str(i)+'个的音') t1 = time.time() player.note_on(Tone[ key_tone[ key_value[i] ] ], volume) elif event.type == pygame.KEYUP:# 按键=松开的话，关闭对应的音调 for i in range(33): if event.key == pygame.__dict__[ 'K_'+key_value[i] ]: print('停止发第'+str(i)+'个') t2 = time.time() t3 = t2 - t1 print(str(t3)+'s') player.note_off(Tone[ key_tone[ key_value[i] ] ], volume)
# 循环获取事件，监听事件状态 for event in pygame.event.get(): # 判断用户是否点了"X"关闭按钮,并执行if代码段 if event.type == pygame.QUIT: #卸载所有模块 print("退出") pygame.quit() #终止程序，确保退出程序 sys.exit() pygame.display.flip() #更新屏幕内容
pygame发声
④窗口显示按键样式和音符：
import pygame.midi import pygame import time import sys # 初始化设置 volume = 127 # 音量 0-127 pygame.init() # 初始化PYgame
windowSurface=pygame.display.set_mode((800,600)) #建立窗口 # screen = pygame.display.set_mode((400,400))
# 设置窗口标题，即游戏名称 pygame.display.set_caption('键盘钢琴')
#引入字体 f = pygame.font.Font('C:/Windows/Fonts/simhei.ttf',75) #生成文本信息，第一个参数文本内容；第二个参数，字体是否平滑； #第三个参数，RGB模式的字体颜色；第四个参数，RGB模式字体背景颜色； text = f.render("Lebron",True,'deeppink','purple') # 获得显示对象的rect区域坐标 textRect = text.get_rect() # 设置显示对象居中 textRect.center = (400,40) # 将准备好的文本信息，绘制到主屏幕 Screen 上。 windowSurface.blit(text,textRect)
device = 0 # device number in win10 laptop instrument = 0 #乐器 http://www.ccarh.org/courses/253/handout/gminstruments/ # initize Pygame MIDI ---------------------------------------------------------- pygame.midi.init()# PYGAMEMIDI库的初始化
Tone = { # 音调字典 'A0':21,'A#0':22,'B0':23, 'C1':24,'C#1':25,'D1':26,'D#1':27,'E1':28,'F1':29,'F#1':30,'G1':31,'G#1':32,'A1':33,'A#1':34,'B1':35, 'C2':36,'C#2':37,'D2':38,'D#2':39,'E2':40,'F2':41,'F#2':42,'G2':43,'G#2':44,'A2':45,'A#2':46,'B2':47, 'C3':48,'C#3':49,'D3':50,'D#3':51,'E3':52,'F3':53,'F#3':54,'G3':55,'G#3':56,'A3':57,'A#3':58,'B3':59, 'C4':60,'C#4':61,'D4':62,'D#4':63,'E4':64,'F4':65,'F#4':66,'G4':67,'G#4':68,'A4':69,'A#4':70,'B4':71, 'C5':72,'C#5':73,'D5':74,'D#5':75,'E5':76,'F5':77,'F#5':78,'G5':79,'G#5':80,'A5':81,'A#5':82,'B5':83, 'C6':84,'C#6':85,'D6':86,'D#6':87,'E6':88,'F6':89,'F#6':90,'G6':91,'G#6':92,'A6':93,'A#6':94,'B6':95, 'C7':96,'C#7':97,'D7':98,'D#7':99,'E7':100,'F7':101,'F#7':102,'G7':103,'G#7':104,'A7':105,'A#7':106,'B7':107, 'C8':108, } # set the output device -------------------------------------------------------- player = pygame.midi.Output(device)#定义了一个输出音轨
# set the instrument ----------------------------------------------------------- player.set_instrument(instrument)#设置乐器音色
key_tone = { '1':"C5", '2':"D5", '3':"E5", '4':"F5", '5':"G5", '6':"A5", '7':"B5", 'q':"C3", 'w':"D3", 'e':"E3", 'r':"F3", 't':"G3", 'y':"A3", 'u':"B3", 'a':"C4", 's':"D4", 'd':"E4", 'f':"F4", 'g':"G4", 'h':"A4", 'j':"B4", 'z':"C2", 'x':"D2", 'c':"E2", 'v':"F2", 'b':"G2", 'n':"A2", 'm':"B2", }
text_col = 'black' # 文本颜色 bd_col1 = 'white' # 背景颜色（初始值） bd_col2 = 'red' # 背景颜色（按下反显值）
def key_color (text,x,y,color): #按键改变颜色（文本内容，坐标x，坐标y，颜色R） key = f.render(text,True,text_col,color) key_rect = key.get_rect() key_rect.center = (x,y) windowSurface.blit(key,key_rect)
for key in key_tone.keys(): #在窗口中循环打印字典中的字符key内容 n=list(key_tone.keys()).index(key) #检索字符 是否在字符串列表中，返回字符所在位置 key_color(key+' ',50+(n//7)*60+(n%7)*90,150+(n//7)*100,bd_col1) # print( list(key_tone.keys()) ) # print( list(key_tone.keys())[0] ) # print(list(key_tone.keys()).index('r'))
while True: for event in pygame.event.get(): # 检测事件 if event.type == pygame.QUIT: #卸载所有模块 print("退出") pygame.quit() #终止程序，确保退出程序 sys.exit() if event.type == pygame.KEYDOWN: if chr(event.key) in key_tone.keys():#所按下的按键，在音符键盘的字典中 n=list(key_tone.keys()).index(chr(event.key))#检索字符 是否在字符串列表中，返回字符所在位置 key_color(chr(event.key)+' ',50+(n//7)*60+(n%7)*90,150+(n//7)*100,bd_col2) print('正在发'+key_tone[chr(event.key)]+'音') t1 = time.time() #记录t1的时间戳 player.note_on(Tone[key_tone[chr(event.key)]], volume) elif event.type == pygame.KEYUP:# 按键=松开的话，关闭对应的音调 if chr(event.key) in key_tone.keys():#所按下的按键，在音符键盘的字典中 n=list(key_tone.keys()).index(chr(event.key))#检索字符 是否在字符串列表中，返回字符所在位置 key_color(chr(event.key)+' ',50+(n//7)*60+(n%7)*90,150+(n//7)*100,bd_col1) print('停止发'+key_tone[chr(event.key)]+'音') t2 = time.time() #记录t2的时间戳 t3 = t2 - t1 #根据t1和t2的时间戳，计算t3时间差 print(str(t3)+'s') #打印出时间差 player.note_off(Tone[key_tone[chr(event.key)]], volume) pygame.display.flip() #更新 time.sleep(0.005) #每0.005s循环一次
窗口显示按键样式和音符
⑤udp通讯：
# -*- coding: utf-8 -*- import socket import time
#client 发送端 client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) PORT = 8008
while True: start = time.time() #获取当前时间 print(time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(start))) #以指定格式显示当前时间 msg=input("本客户端192.168.43.131，请输入要发送的内容：") server_address = ("192.168.43.82", PORT) # 接收方 服务器的ip地址和端口号 client_socket.sendto(bytes(msg.encode("utf-8")), server_address) #将msg内容发送给指定接收方 now = time.time() #获取当前时间 run_time = now-start #计算时间差，即运行时间 print(time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(now))) print("run_time: %d seconds\n" %run_time) time.sleep(1) ###
以上是客户端，以下是服务端
# -*- coding: utf-8 -*- import pygame.midi import socket #导入socket模块 import time #导入time模块
#server 接收端 # 设置服务器默认端口号 PORT = 8008 # 创建一个套接字socket对象，用于进行通讯 # socket.AF_INET 指明使用INET地址集，进行网间通讯 # socket.SOCK_DGRAM 指明使用数据协议，即使用传输层的udp协议 server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) address = ("", PORT) server_socket.bind(address) #为服务器绑定一个固定的地址，ip和端口 server_socket.settimeout(10) #设置一个时间提示，如果10秒钟没接到数据进行提示
device = 0 # device number in win10 laptop instrument = 0 #乐器 http://www.ccarh.org/courses/253/handout/gminstruments/ # initize Pygame MIDI ---------------------------------------------------------- pygame.midi.init()# PYGAMEMIDI库的初始化 # set the output device -------------------------------------------------------- player = pygame.midi.Output(device)#定义了一个输出音轨 volume = 127 # 音量 0-127 # set the instrument ----------------------------------------------------------- player.set_instrument(instrument)#设置乐器音色 Tone = { # 音调字典 'A0':21,'AS0':22,'B0':23, 'C1':24,'CS1':25,'D1':26,'DS1':27,'E1':28,'F1':29,'FS1':30,'G1':31,'GS1':32,'A1':33,'AS1':34,'B1':35, 'C2':36,'CS2':37,'D2':38,'DS2':39,'E2':40,'F2':41,'FS2':42,'G2':43,'GS2':44,'A2':45,'AS2':46,'B2':47, 'C3':48,'CS3':49,'D3':50,'DS3':51,'E3':52,'F3':53,'FS3':54,'G3':55,'GS3':56,'A3':57,'AS3':58,'B3':59, 'C4':60,'CS4':61,'D4':62,'DS4':63,'E4':64,'F4':65,'FS4':66,'G4':67,'GS4':68,'A4':69,'AS4':70,'B4':71, 'C5':72,'CS5':73,'D5':74,'DS5':75,'E5':76,'F5':77,'FS5':78,'G5':79,'GS5':80,'A5':81,'AS5':82,'B5':83, 'C6':84,'CS6':85,'D6':86,'DS6':87,'E6':88,'F6':89,'FS6':90,'G6':91,'GS6':92,'A6':93,'AS6':94,'B6':95, 'C7':96,'CS7':97,'D7':98,'DS7':99,'E7':100,'F7':101,'FS7':102,'G7':103,'GS7':104,'A7':105,'AS7':106,'B7':107, 'C8':108, }
player.note_on(Tone['C3'], volume) time.sleep(1) player.note_off(Tone['C3'], volume)
while True: #正常情况下接收数据并且显示，如果10秒钟没有接收数据进行提示（打印 "time out"） #当然可以不要这个提示，那样的话把"try:" 以及 "except"后的语句删掉就可以了 try: now = time.time() #获取当前时间 # 接收客户端传来的数据 recvfrom接收客户端的数据，默认是阻塞的，直到有客户端传来数据 # recvfrom 参数的意义，表示最大能接收多少数据，单位是字节 # recvfrom返回值说明 # receive_data表示接受到的传来的数据,是bytes类型 # client 表示传来数据的客户端的身份信息，客户端的ip和端口，元组 receive_data, client = server_socket.recvfrom(1024) tone_temp = str(receive_data,'utf-8')#bytes转换为str字符串 if tone_temp in Tone.keys(): player.note_on(Tone[tone_temp], volume) print(time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(now))) #以指定格式显示时间 print("来自客户端%s,发送的%s\n" % (client, receive_data)) #打印接收的内容 except socket.timeout: #如果10秒钟没有接收数据进行提示（打印 "time out"） print("time out")
⑥wifi电子键盘 每个ESP32触摸键盘能够使服务器演奏对应钢琴音：
服务器用pycharm 发送端用thonny
#用thonny 做客户端 pycharm 做服务端 #用thonny 做客户端 pycharm 做服务端 def is_legal_wifi(essid, password): ''' 判断WIFI密码是否合法 ''' if len(essid) == 0 or len(password) == 0: return False return True def do_connect(): import json import network # 尝试读取配置文件wifi_confi.json,这里我们以json的方式来存储WIFI配置 # wifi_config.json在根目录下 # 若不是初次运行,则将文件中的内容读取并加载到字典变量 config try: with open('wifi_config.json','r') as f: config = json.loads(f.read()) # 若初次运行,则将进入excpet,执行配置文件的创建 except: essid = '' password = '' while True: essid = input('wifi name:') # 输入essid password = input('wifi passwrod:') # 输入password if is_legal_wifi(essid, password): config = dict(essid=essid, password=password) # 创建字典 with open('wifi_config.json','w') as f: f.write(json.dumps(config)) # 将字典序列化为json字符串,存入wifi_config.json break else: print('ERROR, Please Input Right WIFI') #以下为正常的WIFI连接流程 wifi = network.WLAN(network.STA_IF) if not wifi.isconnected(): print('connecting to network...') wifi.active(True) wifi.connect(config['essid'], config['password']) import utime for i in range(200): print('第{}次尝试连接WIFI热点'.format(i)) if wifi.isconnected(): break utime.sleep_ms(100) #一般睡个5-10秒,应该绰绰有余 if not wifi.isconnected(): wifi.active(False) #关掉连接,免得repl死循环输出 print('wifi connection error, please reconnect') import os # 连续输错essid和password会导致wifi_config.json不存在 try: os.remove('wifi_config.json') # 删除配置文件 except: pass do_connect() # 重新连接 else: print('network config:', wifi.ifconfig()) import socket import time from machine import Pin from time import sleep LED=Pin(2,Pin.OUT) do_connect() LED.on() #多键触摸发声 #----------------------------------- from machine import TouchPad, Pin, #引用touch库，GPIO库，PWM库 from time import sleep #引用time库 touch_do=TouchPad(Pin(13)) #创建 DO音 TouchPad对象 touch_re=TouchPad(Pin(12)) #创建 RE音 TouchPad对象 touch_mi=TouchPad(Pin(14)) #创建 MI音 TouchPad对象 touch_fa=TouchPad(Pin(27)) #创建 FA音 TouchPad对象 touch_so=TouchPad(Pin(33)) #创建 SO音 TouchPad对象 touch_la=TouchPad(Pin(32)) #创建 LA音 TouchPad对象 touch_si=TouchPad(Pin(15)) #创建 SI音 TouchPad对象 touch_si=TouchPad(Pin(4)) #创建 SI音 TouchPad对象 # LED点亮说明WIFI连接正常 #client 发送端 client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) server_address = ("192.168.43.82", 8008) # 接收方 服务器的ip地址和端口号 while True: #循环体 # A=touch_do.read() # B=touch_re.read() # C=touch_mi.read() # D=touch_fa.read() # E=touch_so.read() # F=touch_la.read() # G=touch_si.read() # H=touch_si.read() PINn = (13,12,14,27,33,32,15,4)#元组 TPx = [TouchPad(Pin(i)) for i in PINn] Tone2 = ( "A3", "A7", "A5", "B3", "D3", "C7", "B5", "A1", ) for ton,tp in zip(Tone2,TPx): if tp.read()<200: print("已经发送啦！") msg = ton client_socket.sendto(bytes(msg.encode('utf-8')), server_address) #将msg内容发送给指定接收方 time.sleep(0.01)
⑦esp32的WiFi连接：
后期这个直接烧录进esp32
import network import time #WIFI连接流程 def wifi_connect(): wifi = network.WLAN(network.STA_IF) wifi.active(True) wifi.connect('', '')
for i in range(200): print('第{}次尝试连接WIFI热点'.format(i)) if wifi.isconnected(): break time.sleep(0.1) #一般睡个5-10秒,应该绰绰有余 if not wifi.isconnected(): wifi.active(False) #关掉连接,免得repl死循环输出 print('wifi connection error, please reconnect') else: print('network config:', wifi.ifconfig())
前端：
①电子琴的完整数据结构 和点击发声：-
<template> <div class="frame"> <div> <!-- 操作区域（退格、清空、播放） --> </div> <div class="voice"> <!-- 这里绘制简谱UI --> </div>
<div class="piano"> <div class="piano_group" v-for="i in 5" :key="i"> <!-- 这里绘制钢琴UI --> <div class="white_keys"> <div class="white-key" v-for="key in piano_data[i-1].white_keys" :key="key" @mousedown="mouseDown(key)" @mouseup="mouseup(key)"> { {key.note}} </div> </div>
<div class="black_keys"> <template v-for="(key,index) in piano_data[i-1].black_keys"> <div :key="index" v-if="key.note==''" class="black_key black_key_empty"></div> <div :key="index" v-else class="black_key" @mousedown="mouseDown(key)" @mouseup="mouseup(key)"></div> </template> <!-- template是对用户隐藏的HTML容器 重复代码-->
</div> </div>
</div> </div> </template>
<script> export default { name: 'Index', components: {}, data() { return { // websocket相关参数 ws: null,
// 基础数据 //白键对应显示 baseWhiteNotes: ['c', 'd', 'e', 'f', 'g', 'a', 'b'], //黑键对应显示 baseBlackNotes: ['cs', 'ds', '', 'fs', 'gs', 'as'], //音符时长 duration: [125, 250, 500, 1000, 2000],
piano_data: [], //时间差 diff_time: 0,
//黑键数据结构 //白键数据结构 //音符键盘对应字典 //乐谱 } }, methods: { // 退格 // 清空乐谱 // 播放乐谱 // 按下琴键 mouseDown(key) { let now = new Date() this.diff_time = now.getTime()
console.log("我按下了") console.log(key)
let audio = document.createElement("audio") audio.src = require("../assets/sound/" + key.note + ".mp3") audio.play()
}, // 松开琴键 mouseup(key) { let now = new Date() this.diff_time = now.getTime() - this.diff_time console.log(this.diff_time)
console.log("我松开了") console.log(key)
let diff = 8888 let diff_index = -1
this.duration.forEach((element, index) => { if (Math.abs(this.diff_time - element) < diff) { diff = Math.abs(this.diff_time - element) diff_index = index }
}); console.log("我按了" + this.diff_time) console.log("我找到距离我最近的音符是:") console.log(this.duration[diff_index])
} }, created() {
for (let i = 0; i < 5; i++){ let group = {
black_keys: [], white_keys: []
}
let white_list = [] this.baseWhiteNotes.forEach((element, index) => { let key = { note: element + (i+2), notenum: index + 1, lh: i+2, half: false
} white_list.push(key) }); group.white_keys = white_list this.piano_data.push(group)
// baseWhiteNotes数字对象（array） element一个对象 完整的forEach let black_List = [] this.baseBlackNotes.forEach((element, index) => { let key = {} key = { note: index == 2 ? "" : element + (i+2), notenum: index + 1, lh: i+2, half: true } black_List.push(key) }); group.black_keys = black_List this.piano_data.push(group)
//构建数据结构 //建立websocket连接 } },
beforeDestroy() { //销毁连接 },
mounted() {} } </script>
<!-- Add "scoped" attribute to limit CSS to this component only --> <style scoped> .frame { display: flex; flex-direction: column; align-items: center; justify-content: flex-start; background-color: rgb(53, 29, 22); height: 100vh; }
.voice { border: 1px solid #eee; box-shadow: 5px 5px 5px #666; width: 620px; /* height: 520px; */ overflow-y: auto; padding: 190px; background-color: wheat; }
.piano { display: flex; align-items: center; justify-content: center; margin-top: 40px; }
.white_keys { display: flex; flex-direction: row; }
.white-key { width: 40px; height: 300px; border: 1px solid #999; /* 边框 border——width宽度 border——style样式 border——color颜色 */ margin-left: -1px; /* 左边距 */ box-shadow: 2px 2px 2px #666; /* 盒子阴影 */ display: flex; align-items: flex-end; justify-content: center; background-color: #FFF; }
.white-key:active { background-color: #666; }
.piano_group { position: relative; }
.black_keys { position: absolute; /* 关于css中的position ，static默认位置，relative相对定位，absolute，fixed固定位置 */ width: calc(100% - 60px); height: 200px; top: 0; left: 25px; display: flex; flex-direction: row; align-items: flex-start; justify-content: space-between; }
.black_key { width: 30px; height: 200px; background-color: #000; z-index: 99; box-shadow: 2px 2px 2px 1px #999; border-block-start: 5px; border-block-end: 5px; } .black_key:active { background-color: red; }
.black_key_empty { z-index: -1; /* 属性指定一个元素的堆叠顺序，为正数会在其上，为负数则在其下 */ } </style>
②建立服务端和前端的websocket连接：
服务端：
from flask import Flask #Flask服务器库 from flask_sockets import Sockets #WS连接库 import pygame.midi #弹奏音乐的库 import pygame #pygame的库 import json from concurrent.futures import ThreadPoolExecutor import socket
instruments = [1,46,25,56]
# 创建线程池执行器 executor = ThreadPoolExecutor(2) # 初始化pygame设置 volume = 127 # 音量 0-127 device = 0 # device number in win10 laptop instrument = 1 # 乐器 http://www.ccarh.org/courses/253/handout/gminstruments/ # initize Pygame MIDI ---------------------------------------------------------- pygame.midi.init() # PYGAMEMIDI库的初始化 # set the output device -------------------------------------------------------- player = pygame.midi.Output(device) # 定义了一个输出音轨 # set the instrument ----------------------------------------------------------- player.set_instrument(instrument) # 设置乐器音色 # 初始化设置结束
Tone = {#音调字典 'C0':12,'CS0':13,'D0':14,'DS0':15,'E0':16,'F0':17,'FS0':18,'G0':19,'GS0':20,'A0':21,'AS0':22,'B0':23, 'C1':24,'CS1':25,'D1':26,'DS1':27,'E1':28,'F1':29,'FS1':30,'G1':31,'GS1':32,'A1':33,'AS1':34,'B1':35, 'C2':36,'CS2':37,'D2':38,'DS2':39,'E2':40,'F2':41,'FS2':42,'G2':43,'GS2':44,'A2':45,'AS2':46,'B2':47, 'C3':48,'CS3':49,'D3':50,'DS3':51,'E3':52,'F3':53,'FS3':54,'G3':55,'GS3':56,'A3':57,'AS3':58,'B3':59, 'C4':60,'CS4':61,'D4':62,'DS4':63,'E4':64,'F4':65,'FS4':66,'G4':67,'GS4':68,'A4':69,'AS4':70,'B4':71, 'C5':72,'CS5':72,'D5':74,'DS5':75,'E5':76,'F5':77,'FS5':78,'G5':79,'GS5':80,'A5':81,'AS5':82,'B5':83, 'C6':84,'CS6':85,'D6':86,'DS6':87,'E6':88,'F6':89,'FS6':90,'G6':91,'GS6':92,'A6':93,'AS6':94,'B6':95, 'C7':96,'CS7':97,'D7':98,'DS7':99,'E7':100,'F7':101,'FS7':102,'G7':103,'GS7':104,'A7':105,'AS7':106,'B7':107, }
wsSev = {} app = Flask(__name__) sockets = Sockets(app)
cheatMode = 0 cheatIndex = 0 currentGroup = ["C4","D4","E4","F4","G4","A4","B4"]
# 服务器启动时，开启ws服务器 def main_app(): from gevent import pywsgi from geventwebsocket.handler import WebSocketHandler server = pywsgi.WSGIServer(('192.168.10.106', 9302), app, handler_class=WebSocketHandler) # 开启udp监听 executor.submit(udp_conn) print("websocket服务启动") server.serve_forever() def udp_conn(): global wsSev global cheatMode global cheatIndex global currentGroup udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) address=('192.168.10.106', 9302) udp_socket.bind(address) print('UDP监听开启') try: while True: revc_data = udp_socket.recvfrom(1024) print(str(revc_data[0], encoding = "utf-8")) noteinfo = json.loads(str(revc_data[0], encoding = "utf-8")) print(cheatMode) if cheatMode == 1 and str(noteinfo["status"])=='1': noteinfo["note"] = currentGroup[cheatIndex] print(currentGroup[cheatIndex]) cheatIndex = cheatIndex + 1 if len(currentGroup) == cheatIndex: cheatIndex = 0 play(noteinfo) if wsSev: wsSev.send(json.dumps(noteinfo)) except: wsTarget.close()
# 持续监听客户端发送的数据 def ws_listener(): global wsSev try: while True: message = wsSev.receive() print(message) if message is not None: noteinfo = json.loads(message) play(noteinfo) #wsSev.send("我接收到了！") except: wsSev.close() # 客户端与服务器建立连接的接口 @sockets.route('/connectServer') def connect_server(socket): global wsSev print('connected') wsSev = socket print('接收到客户端的连接') #wsSev.send("你已成功连接至服务器") ws_listener()
def play(noteinfo): #使用Pygame调用声卡发声 global cheatMode global cheatIndex if str(noteinfo['status']) == "1": print("发声") print(noteinfo['note']) if str(noteinfo['note']) == 'X1': cheatMode = 1 cheatIndex = 0 print("开启cheat模式") elif str(noteinfo['note']) == 'X2': cheatMode = 0 cheatIndex = 0 print("关闭cheat模式") else: player.note_on(Tone[noteinfo['note']], volume) #弹出声音 else: print("停止") player.note_off(Tone[noteinfo['note']], volume) #关闭声音 if __name__ == "__main__": main_app()
前端：
1、访问初始化接口，建立连接
this.ws = new WebSocket("ws://9.7.0.65:9303/connectServer");
ws需要作为一个全局对象，在data中初始化
2、持续监听（接收数据）
this.ws.onmessage = ((event) => {
// 将接收到的数据序列化为JSON结构（Object对象）
let socketMessage = event.data
console.log(socketMessage)
});
3、发送数据
this.ws.send('要发送的内容')icon图标（前端）：
用到UI控件 Element：Element - The world's most popular Vue UI framework
开始前记得，安装和在main.js中引用！
<el-button type="info" circle>播放</el-button> 这样之后，但并不是我想要的结果，接下来有两种方法，第一在button按钮菜单下的图标按钮的下拉代码下选择，第二是icon图标里找寻需要的图标。 图标标签是<i></i> 最终：<el-button type="info" circle><i class="el-icon-caret-right" style="font-size:30px" @click="playSong"></i></el-button>
本项目需要三个icon，第一个：开始播放，第二个：删除前一个音符，第三个删除全部音符
代码如下：
<el-button type="info" circle><i class="el-icon-caret-right" style="font-size:30px" @click="playSong"></i></el-button>
<el-button type="warning" circle><i class="el-icon-back" style="font-size:30px" @click="playSong"></i></el-button>
<el-button type="success" circle><i class="el-icon-close" style="font-size:30px" @click="playSong"></i></el-button>
拓展部分：
①作弊模式：
需要三端联调
硬件端代码如下（只有作弊的代码，不全）：
from machine import Pin import time import socket import json
p0 = Pin(0, Pin.IN) # create input pin on GPIO0 print(p0.value()) # get value, 0 or 1
musickeydict = { "status":"1", "note":"C3" } #client 发送端 client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) server_address = ("9.7.0.65", 9302) # 接收方 服务器的ip地址和端口号
def sendtone(note,status): musickeydict["note"] = note musickeydict["status"] = status client_socket.sendto(bytes(json.dumps(musickeydict).encode('utf-8')), server_address) #将msg内容发送给指定接收方
key_time = 0 while True : time.sleep(0.01) if (p0.value()==0): key_time +=1 if key_time ==3: print("cheat") sendtone('X1','1') else: key_time = 0
服务器：
创建全局变量（下）
# 0：正常模式；1：Cheat模式
cheatMode = 0
# 开启Cheat模式时，对应到乐谱的第几个音
cheatIndex = 0创建乐曲对象 & 导入乐谱（下）
# 创建乐曲对象，以note数组的形式导入
song = ["C4","B3","C4","G4","A4","A4","G4","D4","D4","F4","E4","C4","B3","C4","E4","F4","F4","G4","D4","C4","C4","C4","B3","C4","G4","A4","A4","G4","D4","D4","F4","E4","D4","E4","C4","B3","C4","E4","F4","F4","G4","D4","C4","C4"]开启/关闭Cheat模式
当接收到X1或X2时，开启/关闭Cheat模式
if str(noteinfo['note']) == 'X1':
cheatMode = 1
cheatIndex = 0
print("开启cheat模式")
elif str(noteinfo['note']) == 'X2':
cheatMode = 0
cheatIndex = 0
print("关闭cheat模式")当检测到cheat开启：
if cheatMode == 1 and str(noteinfo["status"])=='1' and str(noteinfo["note"])!='X1' and str(noteinfo["note"])!='X2':
noteinfo["note"] = currentGroup[cheatIndex]
print(currentGroup[cheatIndex])
cheatIndex = cheatIndex + 1
if len(currentGroup) == cheatIndex:
cheatIndex = 0②画面段简谱显示：
<div class="stave"> <div v-for="(item,index) in songs" :key="index" class="note"> <template> <div class="key"> <div v-if="item.lh>0" class="dot"></div> <div v-if="item.lh>1" class="dot"></div> </div> <div class="note-val"> {