Merge pull request #1746 from pipecat-ai/simple_chatbot-react-native
Simple chatbot: React Native client
This commit is contained in:
5
examples/simple-chatbot/client/react-native/.gitignore
vendored
Normal file
5
examples/simple-chatbot/client/react-native/.gitignore
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
ios
|
||||
android
|
||||
node_modules
|
||||
.expo
|
||||
.env
|
||||
1
examples/simple-chatbot/client/react-native/.nvmrc
Normal file
1
examples/simple-chatbot/client/react-native/.nvmrc
Normal file
@@ -0,0 +1 @@
|
||||
22.14
|
||||
21
examples/simple-chatbot/client/react-native/App.js
vendored
Normal file
21
examples/simple-chatbot/client/react-native/App.js
vendored
Normal file
@@ -0,0 +1,21 @@
|
||||
// Disabling the logs from react-native-webrtc
|
||||
import debug from 'debug';
|
||||
debug.disable('rn-webrtc:*');
|
||||
|
||||
// Ignoring the warnings from react-native-background-timer while they don't fix this issue:
|
||||
// https://github.com/ocetnik/react-native-background-timer/issues/366
|
||||
import { LogBox } from 'react-native';
|
||||
LogBox.ignoreLogs([
|
||||
"`new NativeEventEmitter()` was called with a non-null argument without the required `addListener` method.",
|
||||
"`new NativeEventEmitter()` was called with a non-null argument without the required `removeListeners` method."
|
||||
]);
|
||||
|
||||
// Enable debug logs
|
||||
/*window.localStorage = window.localStorage || {};
|
||||
window.localStorage.debug = '*';
|
||||
window.localStorage.getItem = (itemName) => {
|
||||
console.log('Requesting the localStorage item ', itemName);
|
||||
return window.localStorage[itemName];
|
||||
};*/
|
||||
|
||||
export { default } from './src/App';
|
||||
89
examples/simple-chatbot/client/react-native/README.md
Normal file
89
examples/simple-chatbot/client/react-native/README.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# React Native implementation
|
||||
|
||||
Basic implementation using the [Pipecat RN SDK](https://docs.pipecat.ai/client/react-native/introduction).
|
||||
|
||||
## Usage
|
||||
|
||||
### Expo requirements
|
||||
|
||||
This project cannot be used with an [Expo Go](https://docs.expo.dev/workflow/expo-go/) app because [it requires custom native code](https://docs.expo.io/workflow/customizing/).
|
||||
|
||||
When a project requires custom native code or a config plugin, we need to transition from using [Expo Go](https://docs.expo.dev/workflow/expo-go/)
|
||||
to a [development build](https://docs.expo.dev/development/introduction/).
|
||||
|
||||
More details about the custom native code used by this demo can be found in [rn-daily-js-expo-config-plugin](https://github.com/daily-co/rn-daily-js-expo-config-plugin).
|
||||
|
||||
### Building remotely
|
||||
|
||||
If you do not have experience with Xcode and Android Studio builds or do not have them installed locally on your computer, you will need to follow [this guide from Expo to use EAS Build](https://docs.expo.dev/development/create-development-builds/#create-and-install-eas-build).
|
||||
|
||||
### Building locally
|
||||
|
||||
You will need to have installed locally on your computer:
|
||||
- [Xcode](https://developer.apple.com/xcode/) to build for iOS;
|
||||
- [Android Studio](https://developer.android.com/studio) to build for Android;
|
||||
|
||||
#### Install the demo dependencies
|
||||
|
||||
```bash
|
||||
# Use the version of node specified in .nvmrc
|
||||
nvm i
|
||||
|
||||
# Install dependencies
|
||||
yarn install
|
||||
|
||||
# Before a native app can be compiled, the native source code must be generated.
|
||||
npx expo prebuild
|
||||
```
|
||||
|
||||
#### Running on Android
|
||||
|
||||
After plugging in an Android device [configured for debugging](https://developer.android.com/studio/debug/dev-options), run the following command:
|
||||
|
||||
```
|
||||
npm run android
|
||||
```
|
||||
|
||||
#### Running on iOS
|
||||
|
||||
First, you'll need to do a one-time setup. This is required to build to a physical device.
|
||||
|
||||
If you're familiar with Xcode, open `ios/RNSimpleChatbot.xcworkspace` and, in the target settings, provide a development team registered with Apple.
|
||||
|
||||
If you're newer to Xcode, here are some more detailed instructions to get you started.
|
||||
|
||||
First, open the project in Xcode. Make sure to specifically select `RNSimpleChatbot.xcworkspace` from `/ios`. The `/ios` directory will have been generated by running `npx expo prebuild` as instructed above. This is also a good time to plug in your iOS device to be sure the following steps are successful.
|
||||
|
||||
From the main menu, select `Settings` and then `Accounts`. Click the `+` sign to add an account (e.g. an Apple ID).
|
||||
|
||||

|
||||
|
||||
Once an account is added, perform the following steps:
|
||||
|
||||
1. Close `Settings`.
|
||||
1. Select the folder icon in the top left corner.
|
||||
1. Select `RNSimpleChatbot` from the side panel
|
||||
1. Navigate to `Signing & Capabilities` in the top nav bar.
|
||||
1. Open the "Team" dropdown
|
||||
1. Select the account added in the previous step.
|
||||
|
||||
The "Signing Certificate" section should update accordingly with your account information.
|
||||
|
||||

|
||||
|
||||
**Troubleshooting common errors:**
|
||||
|
||||
- If you see the error `Change your bundle identifier to a unique string to try again`, update the "Bundle Identifier" input in `Signing & Capabilities` to make it unique. This should resolve the error.
|
||||
|
||||
- If you see an error that says `Xcode was unable to launch because it has an invalid code signature, inadequate entitlements or its profile has not been explicitly trusted by the user`, you may need to update the settings on your iPhone to enable the required permissions as follows:
|
||||
|
||||
1. Open `Settings` on your iPhone
|
||||
1. Select `General`, then `Device Management`
|
||||
1. Click `Trust` for DailyPlayground
|
||||
|
||||
- You may also be prompted to enter you login keychain password. Be sure to click `Always trust` to avoid the prompt showing multiple times.
|
||||
|
||||
After, run the following command:
|
||||
```
|
||||
npm run ios
|
||||
```
|
||||
75
examples/simple-chatbot/client/react-native/app.json
Normal file
75
examples/simple-chatbot/client/react-native/app.json
Normal file
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"expo": {
|
||||
"name": "RN Simple Chatbot",
|
||||
"slug": "simple-chatbot-demo",
|
||||
"newArchEnabled": false,
|
||||
"version": "1.0.0",
|
||||
"orientation": "portrait",
|
||||
"icon": "./assets/images/pipecat.png",
|
||||
"userInterfaceStyle": "light",
|
||||
"splash": {
|
||||
"image": "./assets/images/splash.png",
|
||||
"resizeMode": "contain",
|
||||
"backgroundColor": "#ffffff"
|
||||
},
|
||||
"updates": {
|
||||
"fallbackToCacheTimeout": 0
|
||||
},
|
||||
"assetBundlePatterns": [
|
||||
"**/*"
|
||||
],
|
||||
"ios": {
|
||||
"supportsTablet": true,
|
||||
"bitcode": false,
|
||||
"bundleIdentifier": "co.daily.SimpleChatbot",
|
||||
"infoPlist": {
|
||||
"UIBackgroundModes": [
|
||||
"voip"
|
||||
]
|
||||
}
|
||||
},
|
||||
"android": {
|
||||
"adaptiveIcon": {
|
||||
"foregroundImage": "./assets/images/pipecat.png",
|
||||
"backgroundColor": "#FFFFFF"
|
||||
},
|
||||
"package": "co.daily.SimpleChatbot",
|
||||
"permissions": [
|
||||
"android.permission.ACCESS_NETWORK_STATE",
|
||||
"android.permission.BLUETOOTH",
|
||||
"android.permission.CAMERA",
|
||||
"android.permission.INTERNET",
|
||||
"android.permission.MODIFY_AUDIO_SETTINGS",
|
||||
"android.permission.RECORD_AUDIO",
|
||||
"android.permission.SYSTEM_ALERT_WINDOW",
|
||||
"android.permission.WAKE_LOCK",
|
||||
"android.permission.FOREGROUND_SERVICE",
|
||||
"android.permission.FOREGROUND_SERVICE_CAMERA",
|
||||
"android.permission.FOREGROUND_SERVICE_MICROPHONE",
|
||||
"android.permission.FOREGROUND_SERVICE_MEDIA_PROJECTION",
|
||||
"android.permission.POST_NOTIFICATIONS"
|
||||
]
|
||||
},
|
||||
"web": {
|
||||
"favicon": "./assets/images/pipecat.png"
|
||||
},
|
||||
"plugins": [
|
||||
"@config-plugins/react-native-webrtc",
|
||||
"@daily-co/config-plugin-rn-daily-js",
|
||||
[
|
||||
"expo-build-properties",
|
||||
{
|
||||
"android": {
|
||||
"minSdkVersion": 24,
|
||||
"compileSdkVersion": 35,
|
||||
"targetSdkVersion": 35,
|
||||
"buildToolsVersion": "35.0.0"
|
||||
},
|
||||
"ios": {
|
||||
"deploymentTarget": "15.1"
|
||||
}
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 3.7 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 29 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 46 KiB |
@@ -0,0 +1,6 @@
|
||||
module.exports = function(api) {
|
||||
api.cache(true);
|
||||
return {
|
||||
presets: ['babel-preset-expo'],
|
||||
};
|
||||
};
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 177 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 213 KiB |
1
examples/simple-chatbot/client/react-native/env.example
Normal file
1
examples/simple-chatbot/client/react-native/env.example
Normal file
@@ -0,0 +1 @@
|
||||
EXPO_SIMPLE_CHATBOT_SERVER=http://$YOUR_IP:7860
|
||||
@@ -0,0 +1,2 @@
|
||||
const { getDefaultConfig } = require('expo/metro-config');
|
||||
module.exports = getDefaultConfig(__dirname);
|
||||
44
examples/simple-chatbot/client/react-native/package.json
Normal file
44
examples/simple-chatbot/client/react-native/package.json
Normal file
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"name": "simple-chatbot-demo",
|
||||
"version": "1.0.0",
|
||||
"scripts": {
|
||||
"start": "expo start --dev-client",
|
||||
"android": "expo run:android --device",
|
||||
"ios": "expo run:ios --device",
|
||||
"web": "expo start --web",
|
||||
"update": "(cd ../rtvi-client-react-native-daily; yarn build); cp -R ../rtvi-client-react-native-daily/lib/* ./node_modules/react-native-realtime-ai-daily/lib/;"
|
||||
},
|
||||
"dependencies": {
|
||||
"@config-plugins/react-native-webrtc": "^10.0.0",
|
||||
"@daily-co/config-plugin-rn-daily-js": "0.0.7",
|
||||
"@daily-co/react-native-daily-js": "^0.76.0",
|
||||
"@daily-co/react-native-webrtc": "^118.0.3-daily.2",
|
||||
"@react-native-async-storage/async-storage": "1.24.0",
|
||||
"@react-navigation/native": "^7.0.14",
|
||||
"@react-navigation/stack": "^7.1.1",
|
||||
"expo": "^53.0.7",
|
||||
"expo-build-properties": "~0.14.6",
|
||||
"expo-dev-client": "~5.1.8",
|
||||
"expo-splash-screen": "~0.30.8",
|
||||
"expo-status-bar": "~2.2.3",
|
||||
"react": "19.0.0",
|
||||
"react-native": "0.79.2",
|
||||
"react-native-background-timer": "^2.4.1",
|
||||
"react-native-gesture-handler": "^2.25.0",
|
||||
"react-native-get-random-values": "^1.11.0",
|
||||
"@pipecat-ai/react-native-daily-transport": "^0.3.5",
|
||||
"react-native-safe-area-context": "^5.4.0",
|
||||
"react-native-screens": "^4.10.0",
|
||||
"react-native-toast-message": "^2.3.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@babel/core": "^7.27.1",
|
||||
"@types/react-native": "^0.73.0",
|
||||
"typescript": "~5.8.3"
|
||||
},
|
||||
"private": true,
|
||||
"resolutions": {
|
||||
"@daily-co/react-native-webrtc/debug": "^4.0.0",
|
||||
"@daily-co/react-native-webrtc/@types/react-native": "^0.73.0"
|
||||
}
|
||||
}
|
||||
34
examples/simple-chatbot/client/react-native/src/App.tsx
Normal file
34
examples/simple-chatbot/client/react-native/src/App.tsx
Normal file
@@ -0,0 +1,34 @@
|
||||
import React from "react"
|
||||
|
||||
import { NavigationContainer } from '@react-navigation/native';
|
||||
import { createStackNavigator } from '@react-navigation/stack';
|
||||
import PreJoinView from './views/PreJoinView';
|
||||
import MeetingView from './views/MeetingView';
|
||||
import { VoiceClientProvider } from './context/VoiceClientContext';
|
||||
import Toast from 'react-native-toast-message';
|
||||
|
||||
import { useVoiceClientNavigation } from './hooks/useVoiceClientNavigation';
|
||||
|
||||
const Stack = createStackNavigator();
|
||||
|
||||
const NavigationManager: React.FC = () => {
|
||||
useVoiceClientNavigation(); // This hook now controls the navigation based on the connection state.
|
||||
return null; // This component doesn't render anything but manages navigation.
|
||||
};
|
||||
|
||||
const App: React.FC = () => {
|
||||
return (
|
||||
<VoiceClientProvider>
|
||||
<NavigationContainer>
|
||||
<Stack.Navigator initialRouteName="Prejoin">
|
||||
<Stack.Screen name="Prejoin" component={PreJoinView} options={{ headerShown: false }}/>
|
||||
<Stack.Screen name="Meeting" component={MeetingView} options={{ headerShown: false }}/>
|
||||
</Stack.Navigator>
|
||||
<NavigationManager />
|
||||
<Toast />
|
||||
</NavigationContainer>
|
||||
</VoiceClientProvider>
|
||||
);
|
||||
};
|
||||
|
||||
export default App;
|
||||
@@ -0,0 +1,94 @@
|
||||
import React, { useState, useMemo } from 'react';
|
||||
import { View, StyleSheet, LayoutChangeEvent, ViewStyle } from 'react-native';
|
||||
import { MaterialIcons } from '@expo/vector-icons';
|
||||
import Colors from '../theme/Colors';
|
||||
import { useVoiceClient } from '../context/VoiceClientContext';
|
||||
|
||||
interface MicrophoneViewProps {
|
||||
style?: ViewStyle; // Optional additional styles for the button container
|
||||
}
|
||||
|
||||
const MicrophoneView: React.FC<MicrophoneViewProps> = ({ style }) => {
|
||||
const { isMicEnabled, localAudioLevel: audioLevel } = useVoiceClient();
|
||||
const [dimensions, setDimensions] = useState({ width: 0, height: 0 });
|
||||
|
||||
const onLayout = (event: LayoutChangeEvent) => {
|
||||
const { width, height } = event.nativeEvent.layout;
|
||||
setDimensions({ width, height });
|
||||
};
|
||||
|
||||
const { width } = dimensions;
|
||||
|
||||
const circleSize = useMemo(() => width * 0.9, [width]);
|
||||
const innerCircleSize = useMemo(() => width * 0.82, [width]);
|
||||
const audioCircleSize = useMemo(() => audioLevel * width * 0.95, [audioLevel, width]);
|
||||
|
||||
return (
|
||||
<View style={[styles.container, style]} onLayout={onLayout}>
|
||||
<View
|
||||
style={[
|
||||
styles.outerCircle,
|
||||
{ width: circleSize, height: circleSize, borderRadius: circleSize / 2 },
|
||||
]}
|
||||
>
|
||||
<View
|
||||
style={[
|
||||
styles.innerCircle,
|
||||
{
|
||||
backgroundColor: !isMicEnabled ? Colors.disabledMic : Colors.backgroundCircle,
|
||||
width: innerCircleSize,
|
||||
height: innerCircleSize,
|
||||
borderRadius: innerCircleSize / 2,
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
{isMicEnabled && (
|
||||
<View
|
||||
style={[
|
||||
styles.audioCircle,
|
||||
{
|
||||
width: audioCircleSize,
|
||||
height: audioCircleSize,
|
||||
borderRadius: audioCircleSize / 2,
|
||||
},
|
||||
]}
|
||||
/>
|
||||
)}
|
||||
|
||||
<MaterialIcons
|
||||
name={!isMicEnabled ? "mic-off" : "mic"}
|
||||
size={width * 0.2}
|
||||
color="white"
|
||||
style={styles.micIcon}
|
||||
/>
|
||||
</View>
|
||||
</View>
|
||||
);
|
||||
};
|
||||
|
||||
const styles = StyleSheet.create({
|
||||
container: {
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
} as ViewStyle,
|
||||
outerCircle: {
|
||||
borderWidth: 1,
|
||||
borderColor: Colors.buttonsBorder,
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
} as ViewStyle,
|
||||
innerCircle: {
|
||||
position: 'absolute',
|
||||
} as ViewStyle,
|
||||
audioCircle: {
|
||||
position: 'absolute',
|
||||
backgroundColor: Colors.micVolume,
|
||||
opacity: 0.5,
|
||||
} as ViewStyle,
|
||||
micIcon: {
|
||||
position: 'absolute',
|
||||
},
|
||||
});
|
||||
|
||||
export default MicrophoneView;
|
||||
@@ -0,0 +1,128 @@
|
||||
import React, { useEffect, useState } from 'react';
|
||||
import { LayoutChangeEvent, StyleSheet, Text, View, ViewStyle } from 'react-native';
|
||||
import { MaterialIcons } from '@expo/vector-icons';
|
||||
import Colors from '../theme/Colors';
|
||||
import { useVoiceClient } from '../context/VoiceClientContext';
|
||||
|
||||
const dotCount = 5;
|
||||
|
||||
const WaveformView: React.FC = () => {
|
||||
const [audioLevels, setAudioLevels] = useState(Array(dotCount).fill(0));
|
||||
const [dimensions, setDimensions] = useState({ width: 0, height: 0 });
|
||||
|
||||
const { currentState: voiceClientStatus, botReady: isBotReady, remoteAudioLevel: audioLevel } = useVoiceClient();
|
||||
|
||||
const onLayout = (event: LayoutChangeEvent) => {
|
||||
const { width, height } = event.nativeEvent.layout;
|
||||
setDimensions({ width, height });
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
setAudioLevels((prevLevels) => [...prevLevels.slice(1), audioLevel]);
|
||||
}, [audioLevel]);
|
||||
|
||||
const { width, height } = dimensions;
|
||||
const circleSize = width * 0.9;
|
||||
const innerCircleSize = width * 0.82;
|
||||
const barWidth = (width * 0.5) / dotCount;
|
||||
|
||||
return (
|
||||
<View style={styles.container} onLayout={onLayout}>
|
||||
<View style={[styles.outerCircle, { width: circleSize, height: circleSize, borderRadius: circleSize / 2 }]}>
|
||||
<View
|
||||
style={[
|
||||
styles.innerCircle,
|
||||
{
|
||||
backgroundColor: isBotReady ? Colors.backgroundCircle : Colors.backgroundCircleNotConnected,
|
||||
width: innerCircleSize,
|
||||
height: innerCircleSize,
|
||||
borderRadius: innerCircleSize / 2,
|
||||
},
|
||||
]}
|
||||
>
|
||||
{isBotReady ? (
|
||||
audioLevel > 0 ? (
|
||||
<View style={[styles.waveformContainer, { width: width * 0.5, height: width * 0.5 }]}>
|
||||
{audioLevels.map((level, index) => (
|
||||
<View
|
||||
key={index}
|
||||
style={[
|
||||
styles.waveformBar,
|
||||
{
|
||||
width: barWidth - 10, // Subtract some margin
|
||||
height: level * height,
|
||||
},
|
||||
]}
|
||||
/>
|
||||
))}
|
||||
</View>
|
||||
) : (
|
||||
<View style={[styles.dotContainer, { width: width * 0.5, height: height * 0.5 }]}>
|
||||
{Array(dotCount)
|
||||
.fill(0)
|
||||
.map((_, index) => (
|
||||
<View key={index} style={[styles.dot, { width: height * 0.1, height: height * 0.1 }]} />
|
||||
))}
|
||||
</View>
|
||||
)
|
||||
) : (
|
||||
<View style={styles.notReadyContainer}>
|
||||
<MaterialIcons name="hourglass-empty" size={32} color="white" />
|
||||
<Text style={styles.voiceClientStatusText}>{voiceClientStatus}</Text>
|
||||
</View>
|
||||
)}
|
||||
</View>
|
||||
</View>
|
||||
</View>
|
||||
);
|
||||
};
|
||||
|
||||
const styles = StyleSheet.create({
|
||||
container: {
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
width: "100%",
|
||||
} as ViewStyle,
|
||||
outerCircle: {
|
||||
borderWidth: 1,
|
||||
borderColor: 'gray',
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
} as ViewStyle,
|
||||
innerCircle: {
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
position: 'relative',
|
||||
} as ViewStyle,
|
||||
waveformContainer: {
|
||||
flexDirection: 'row',
|
||||
justifyContent: 'space-between',
|
||||
alignItems: 'center',
|
||||
} as ViewStyle,
|
||||
waveformBar: {
|
||||
backgroundColor: 'white',
|
||||
maxHeight: '100%',
|
||||
borderRadius: 12,
|
||||
} as ViewStyle,
|
||||
dotContainer: {
|
||||
flexDirection: 'row',
|
||||
justifyContent: 'space-between',
|
||||
alignItems: 'center',
|
||||
} as ViewStyle,
|
||||
dot: {
|
||||
backgroundColor: 'white',
|
||||
borderRadius: 50,
|
||||
} as ViewStyle,
|
||||
notReadyContainer: {
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
} as ViewStyle,
|
||||
voiceClientStatusText: {
|
||||
color: 'white',
|
||||
marginTop: 10,
|
||||
fontSize: 16,
|
||||
fontWeight: 'bold',
|
||||
} as ViewStyle,
|
||||
});
|
||||
|
||||
export default WaveformView;
|
||||
@@ -0,0 +1,229 @@
|
||||
import React, { createContext, useState, useContext, ReactNode, useCallback, useMemo, useRef, useEffect } from 'react'
|
||||
import Toast from 'react-native-toast-message'
|
||||
import { RNDailyTransport } from '@pipecat-ai/react-native-daily-transport'
|
||||
import { RTVIClient, TransportState, RTVIMessage, Participant } from '@pipecat-ai/client-js'
|
||||
import { MediaStreamTrack } from '@daily-co/react-native-webrtc'
|
||||
import { SettingsManager } from '../settings/SettingsManager';
|
||||
|
||||
interface VoiceClientContextProps {
|
||||
voiceClient: RTVIClient | null
|
||||
inCall: boolean
|
||||
currentState: string
|
||||
botReady: boolean
|
||||
localAudioLevel: number
|
||||
remoteAudioLevel: number
|
||||
isMicEnabled: boolean
|
||||
isCamEnabled: boolean
|
||||
videoTrack?: MediaStreamTrack
|
||||
timerCountDown: number
|
||||
// methods
|
||||
start: (url: string) => Promise<void>
|
||||
leave: () => void
|
||||
toggleMicInput: () => void
|
||||
toggleCamInput: () => void
|
||||
}
|
||||
|
||||
export const VoiceClientContext = createContext<VoiceClientContextProps | undefined>(undefined)
|
||||
|
||||
interface VoiceClientProviderProps {
|
||||
children: ReactNode
|
||||
}
|
||||
|
||||
export const VoiceClientProvider: React.FC<VoiceClientProviderProps> = ({ children }) => {
|
||||
const [voiceClient, setVoiceClient] = useState<RTVIClient | null>(null)
|
||||
const [inCall, setInCall] = useState<boolean>(false)
|
||||
const [currentState, setCurrentState] = useState<TransportState>("disconnected")
|
||||
const [botReady, setBotReady] = useState<boolean>(false)
|
||||
const [isMicEnabled, setIsMicEnabled] = useState<boolean>(false)
|
||||
const [isCamEnabled, setIsCamEnabled] = useState<boolean>(false)
|
||||
const [videoTrack, setVideoTrack] = useState<MediaStreamTrack>()
|
||||
const [localAudioLevel, setLocalAudioLevel] = useState<number>(0)
|
||||
const [remoteAudioLevel, setRemoteAudioLevel] = useState<number>(0)
|
||||
const [timerCountDown, setTimerCountDown] = useState<number>(0)
|
||||
|
||||
const botSpeakingRef = useRef(false)
|
||||
let meetingTimer: NodeJS.Timeout | null
|
||||
|
||||
const createVoiceClient = useCallback((url: string): RTVIClient => {
|
||||
return new RTVIClient({
|
||||
transport: new RNDailyTransport(),
|
||||
params: {
|
||||
baseUrl: url,
|
||||
endpoints: {
|
||||
connect: "/connect"
|
||||
}
|
||||
},
|
||||
enableMic: true,
|
||||
enableCam: false
|
||||
})
|
||||
}, [])
|
||||
|
||||
const handleError = useCallback((error: any) => {
|
||||
console.log("Error occurred:", error)
|
||||
const errorMessage = error.message || error.data?.error || "An unexpected error occurred"
|
||||
Toast.show({
|
||||
type: 'error',
|
||||
text1: errorMessage,
|
||||
})
|
||||
}, [])
|
||||
|
||||
const setupListeners = useCallback((voiceClient: RTVIClient): void => {
|
||||
const inCallStates = new Set(["authenticating", "connecting", "connected", "ready"])
|
||||
|
||||
voiceClient
|
||||
.on("transportStateChanged", (state: TransportState) => {
|
||||
setCurrentState(voiceClient.state)
|
||||
setInCall(inCallStates.has(state))
|
||||
})
|
||||
.on("error", (error: RTVIMessage) => {
|
||||
handleError(error)
|
||||
})
|
||||
.on("botReady", () => {
|
||||
setBotReady(true)
|
||||
let expirationTime = voiceClient.transportExpiry
|
||||
if (expirationTime) {
|
||||
startTimer(expirationTime)
|
||||
}
|
||||
})
|
||||
.on("disconnected", () => {
|
||||
setBotReady(false)
|
||||
stopTimer()
|
||||
setIsMicEnabled(false)
|
||||
setIsCamEnabled(false)
|
||||
})
|
||||
.on("localAudioLevel", (level: number) => {
|
||||
setLocalAudioLevel(level)
|
||||
})
|
||||
.on("remoteAudioLevel", (level: number) => {
|
||||
if (botSpeakingRef.current) {
|
||||
setRemoteAudioLevel(level)
|
||||
}
|
||||
})
|
||||
.on("userStartedSpeaking", () => {
|
||||
// nothing to do here
|
||||
})
|
||||
.on("userStoppedSpeaking", () => {
|
||||
// nothing to do here
|
||||
})
|
||||
.on("botStartedSpeaking", () => {
|
||||
botSpeakingRef.current = true
|
||||
})
|
||||
.on("botStoppedSpeaking", () => {
|
||||
botSpeakingRef.current = false
|
||||
setRemoteAudioLevel(0)
|
||||
})
|
||||
.on("connected", () => {
|
||||
setIsMicEnabled(voiceClient.isMicEnabled)
|
||||
setIsCamEnabled(voiceClient.isCamEnabled)
|
||||
})
|
||||
.on("trackStarted", (track: MediaStreamTrack, p?: Participant) => {
|
||||
if (p?.local && track.kind === 'video'){
|
||||
setVideoTrack(track)
|
||||
}
|
||||
})
|
||||
}, [handleError])
|
||||
|
||||
const start = useCallback(async (url: string): Promise<void> => {
|
||||
const client = createVoiceClient(url)
|
||||
setVoiceClient(client)
|
||||
setupListeners(client)
|
||||
try {
|
||||
await client.connect()
|
||||
// updating the preferences
|
||||
const newSettings = await SettingsManager.getSettings();
|
||||
newSettings.backendURL = url
|
||||
await SettingsManager.updateSettings(newSettings)
|
||||
} catch (error) {
|
||||
handleError(error)
|
||||
}
|
||||
}, [createVoiceClient, setupListeners, handleError])
|
||||
|
||||
const leave = useCallback(async (): Promise<void> => {
|
||||
if (voiceClient) {
|
||||
await voiceClient.disconnect()
|
||||
setVoiceClient(null)
|
||||
}
|
||||
}, [voiceClient])
|
||||
|
||||
const toggleMicInput = useCallback(async (): Promise<void> => {
|
||||
if (voiceClient) {
|
||||
try {
|
||||
let enableMic = !isMicEnabled
|
||||
voiceClient.enableMic(enableMic)
|
||||
setIsMicEnabled(enableMic)
|
||||
} catch (e) {
|
||||
handleError(e)
|
||||
}
|
||||
}
|
||||
}, [voiceClient, isMicEnabled])
|
||||
|
||||
const toggleCamInput = useCallback(async (): Promise<void> => {
|
||||
if (voiceClient) {
|
||||
try {
|
||||
let enableCam = !isCamEnabled
|
||||
voiceClient.enableCam(enableCam)
|
||||
setIsCamEnabled(enableCam)
|
||||
} catch (e) {
|
||||
handleError(e)
|
||||
}
|
||||
}
|
||||
}, [voiceClient, isCamEnabled])
|
||||
|
||||
const startTimer = (expirationTime: number): void => {
|
||||
const currentTime = Math.floor(Date.now() / 1000)
|
||||
const leftTime = expirationTime - currentTime
|
||||
setTimerCountDown(leftTime)
|
||||
meetingTimer = setInterval(() => {
|
||||
setTimerCountDown((prevCountDown) => {
|
||||
return prevCountDown - 1
|
||||
})
|
||||
}, 1000)
|
||||
}
|
||||
|
||||
const stopTimer = (): void => {
|
||||
if (meetingTimer) {
|
||||
clearInterval(meetingTimer)
|
||||
meetingTimer = null
|
||||
}
|
||||
setTimerCountDown(0)
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
if (voiceClient) {
|
||||
voiceClient.removeAllListeners() // Cleanup on unmount
|
||||
}
|
||||
}
|
||||
}, [voiceClient])
|
||||
|
||||
const contextValue = useMemo(() => ({
|
||||
voiceClient,
|
||||
inCall,
|
||||
currentState,
|
||||
botReady,
|
||||
isMicEnabled,
|
||||
isCamEnabled,
|
||||
localAudioLevel,
|
||||
remoteAudioLevel,
|
||||
videoTrack,
|
||||
timerCountDown,
|
||||
start,
|
||||
leave,
|
||||
toggleMicInput,
|
||||
toggleCamInput
|
||||
}), [voiceClient, inCall, currentState, botReady, isMicEnabled, isCamEnabled, localAudioLevel, remoteAudioLevel, videoTrack, timerCountDown, start, leave, toggleMicInput, toggleCamInput])
|
||||
|
||||
return (
|
||||
<VoiceClientContext.Provider value={contextValue}>
|
||||
{children}
|
||||
</VoiceClientContext.Provider>
|
||||
)
|
||||
}
|
||||
|
||||
export const useVoiceClient = (): VoiceClientContextProps => {
|
||||
const context = useContext(VoiceClientContext)
|
||||
if (!context) {
|
||||
throw new Error('useVoiceClient must be used within a VoiceClientProvider')
|
||||
}
|
||||
return context
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
import { useEffect } from 'react';
|
||||
import { useNavigation, NavigationProp } from '@react-navigation/native';
|
||||
import { useVoiceClient } from '../context/VoiceClientContext';
|
||||
|
||||
export type RootStackParamList = {
|
||||
Meeting: undefined;
|
||||
Prejoin: undefined;
|
||||
};
|
||||
|
||||
export const useVoiceClientNavigation = () => {
|
||||
const navigation = useNavigation<NavigationProp<RootStackParamList>>();
|
||||
const { inCall } = useVoiceClient();
|
||||
|
||||
useEffect(() => {
|
||||
if (inCall) {
|
||||
navigation.navigate('Meeting');
|
||||
} else {
|
||||
navigation.navigate('Prejoin');
|
||||
}
|
||||
}, [inCall, navigation]);
|
||||
|
||||
};
|
||||
@@ -0,0 +1,42 @@
|
||||
import AsyncStorage from '@react-native-async-storage/async-storage';
|
||||
|
||||
export interface SettingsManager {
|
||||
enableCam: boolean;
|
||||
enableMic: boolean;
|
||||
backendURL: string;
|
||||
}
|
||||
|
||||
// Define the settings object
|
||||
const defaultSettings: SettingsManager = {
|
||||
enableCam: false,
|
||||
enableMic: true,
|
||||
backendURL: process.env.EXPO_SIMPLE_CHATBOT_SERVER || "",
|
||||
};
|
||||
|
||||
export class SettingsManager {
|
||||
private static preferencesKey = 'settingsPreference';
|
||||
|
||||
static async getSettings(): Promise<SettingsManager> {
|
||||
try {
|
||||
const data = await AsyncStorage.getItem(this.preferencesKey);
|
||||
if (data !== null) {
|
||||
return JSON.parse(data) as SettingsManager;
|
||||
} else {
|
||||
return defaultSettings;
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to load settings:", error);
|
||||
return defaultSettings;
|
||||
}
|
||||
}
|
||||
|
||||
static async updateSettings(settings: SettingsManager): Promise<void> {
|
||||
try {
|
||||
const data = JSON.stringify(settings);
|
||||
await AsyncStorage.setItem(this.preferencesKey, data);
|
||||
} catch (error) {
|
||||
console.error("Failed to save settings:", error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
export const Images = {
|
||||
dailyBot: require('../../assets/images/pipecat.png'),
|
||||
};
|
||||
|
||||
export const Icons = {
|
||||
vision: require('../../assets/icons/vision.png'),
|
||||
};
|
||||
@@ -0,0 +1,27 @@
|
||||
type ColorsType = {
|
||||
white: string;
|
||||
black: string;
|
||||
backgroundCircle: string;
|
||||
backgroundCircleNotConnected: string;
|
||||
backgroundApp: string;
|
||||
buttonsBorder: string;
|
||||
micVolume: string;
|
||||
timer: string;
|
||||
disabledMic: string;
|
||||
disabledVision: string;
|
||||
};
|
||||
|
||||
const Colors: ColorsType = {
|
||||
white: '#ffffff',
|
||||
black: '#000000',
|
||||
backgroundCircle: '#374151',
|
||||
backgroundCircleNotConnected: '#D1D5DB',
|
||||
backgroundApp: '#F9FAFB',
|
||||
buttonsBorder: '#E5E7EB',
|
||||
micVolume: '#86EFAC',
|
||||
timer: '#E5E7EB',
|
||||
disabledMic: '#ee6b6e',
|
||||
disabledVision: '#BBF7D0',
|
||||
};
|
||||
|
||||
export default Colors;
|
||||
@@ -0,0 +1,62 @@
|
||||
import React from 'react';
|
||||
import { TouchableOpacity, Text, StyleSheet, ViewStyle, TextStyle, GestureResponderEvent } from 'react-native';
|
||||
import { MaterialIcons } from '@expo/vector-icons';
|
||||
|
||||
interface CustomButtonProps {
|
||||
title: string;
|
||||
onPress: (event: GestureResponderEvent) => void;
|
||||
backgroundColor?: string; // Optional prop for background color
|
||||
textColor?: string; // Optional prop for text color
|
||||
style?: ViewStyle; // Optional additional styles for the button container
|
||||
textStyle?: TextStyle; // Optional additional styles for the text
|
||||
iconName?: string; // Optional prop for the icon name from MaterialIcons
|
||||
iconPosition?: 'left' | 'right'; // Optional prop to control icon position
|
||||
iconSize?: number; // Optional prop for icon size
|
||||
iconColor?: string; // Optional prop for icon color
|
||||
}
|
||||
|
||||
const CustomButton: React.FC<CustomButtonProps> = ({
|
||||
title,
|
||||
onPress,
|
||||
backgroundColor = 'black',
|
||||
textColor = 'white',
|
||||
style,
|
||||
textStyle,
|
||||
iconName,
|
||||
iconPosition = 'left',
|
||||
iconSize = 24,
|
||||
iconColor = 'white',
|
||||
}) => {
|
||||
return (
|
||||
<TouchableOpacity
|
||||
onPress={onPress}
|
||||
style={[styles.button, { backgroundColor }, style]}>
|
||||
{iconName && iconPosition === 'left' && (
|
||||
<MaterialIcons name={iconName as keyof typeof MaterialIcons.glyphMap} size={iconSize} color={iconColor} style={styles.icon} />
|
||||
)}
|
||||
<Text style={[styles.text, { color: textColor }, textStyle]}>{title}</Text>
|
||||
{iconName && iconPosition === 'right' && (
|
||||
<MaterialIcons name={iconName as keyof typeof MaterialIcons.glyphMap} size={iconSize} color={iconColor} style={styles.icon} />
|
||||
)}
|
||||
</TouchableOpacity>
|
||||
);
|
||||
};
|
||||
|
||||
const styles = StyleSheet.create({
|
||||
button: {
|
||||
padding: 12,
|
||||
borderRadius: 8,
|
||||
alignItems: 'center',
|
||||
justifyContent: 'center',
|
||||
flexDirection: 'row', // Ensures icon and text are aligned in a row
|
||||
},
|
||||
text: {
|
||||
fontSize: 16,
|
||||
fontWeight: 'bold',
|
||||
},
|
||||
icon: {
|
||||
marginHorizontal: 5, // Adds space between the icon and text
|
||||
},
|
||||
});
|
||||
|
||||
export default CustomButton;
|
||||
@@ -0,0 +1,139 @@
|
||||
import {
|
||||
View,
|
||||
StyleSheet,
|
||||
Text,
|
||||
Image,
|
||||
TouchableOpacity,
|
||||
} from 'react-native';
|
||||
|
||||
import React from "react"
|
||||
|
||||
import { useVoiceClient } from '../context/VoiceClientContext';
|
||||
|
||||
import { Images } from '../theme/Assets';
|
||||
import { MaterialIcons } from '@expo/vector-icons';
|
||||
|
||||
import WaveformView from '../components/WaveformView';
|
||||
import MicrophoneView from '../components/MicrophoneView';
|
||||
import { SafeAreaView } from 'react-native-safe-area-context';
|
||||
import Colors from '../theme/Colors';
|
||||
import CustomButton from '../theme/CustomButton';
|
||||
|
||||
const MeetingView: React.FC = () => {
|
||||
|
||||
const { leave, toggleMicInput, toggleCamInput, timerCountDown } = useVoiceClient();
|
||||
|
||||
const timerString = (count: number): string => {
|
||||
const hours = Math.floor(count / 3600);
|
||||
const minutes = Math.floor((count % 3600) / 60);
|
||||
const seconds = count % 60;
|
||||
return `${String(hours).padStart(2, '0')}:${String(minutes).padStart(2, '0')}:${String(seconds).padStart(2, '0')}`;
|
||||
};
|
||||
|
||||
return (
|
||||
<SafeAreaView style={styles.safeArea}>
|
||||
<View style={styles.container}>
|
||||
<View style={styles.header}>
|
||||
<Image source={Images.dailyBot} style={styles.botImage} />
|
||||
<View style={styles.timerContainer}>
|
||||
<MaterialIcons name="timelapse" size={24} color="black" />
|
||||
<Text style={styles.timerText}>{timerString(timerCountDown)}</Text>
|
||||
</View>
|
||||
</View>
|
||||
|
||||
<View style={styles.mainPanel}>
|
||||
<WaveformView/>
|
||||
<View style={styles.bottomControls}>
|
||||
<TouchableOpacity onPress={toggleMicInput}>
|
||||
<MicrophoneView
|
||||
style={styles.microphone}
|
||||
/>
|
||||
</TouchableOpacity>
|
||||
</View>
|
||||
</View>
|
||||
|
||||
{/* Bottom Panel */}
|
||||
<View style={styles.bottomPanel}>
|
||||
<CustomButton
|
||||
title="End"
|
||||
iconName={"exit-to-app"}
|
||||
onPress={leave}
|
||||
backgroundColor={Colors.black}
|
||||
/>
|
||||
</View>
|
||||
</View>
|
||||
</SafeAreaView>
|
||||
);
|
||||
};
|
||||
|
||||
const styles = StyleSheet.create({
|
||||
safeArea: {
|
||||
flex: 1,
|
||||
width: "100%",
|
||||
backgroundColor: Colors.backgroundApp,
|
||||
},
|
||||
container: {
|
||||
flex: 1,
|
||||
padding: 20,
|
||||
},
|
||||
header: {
|
||||
flexDirection: 'row',
|
||||
alignItems: 'center',
|
||||
justifyContent: 'space-between',
|
||||
paddingBottom: 10,
|
||||
},
|
||||
botImage: {
|
||||
width: 48,
|
||||
height: 48,
|
||||
},
|
||||
timerContainer: {
|
||||
flexDirection: 'row',
|
||||
alignItems: 'center',
|
||||
backgroundColor: Colors.timer,
|
||||
padding: 10,
|
||||
borderRadius: 12,
|
||||
},
|
||||
timerText: {
|
||||
color: 'black',
|
||||
fontWeight: '500',
|
||||
fontSize: 18,
|
||||
marginLeft: 5,
|
||||
},
|
||||
mainPanel: {
|
||||
flex: 1,
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
},
|
||||
bottomControls: {
|
||||
flexDirection: 'row',
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
width: '100%',
|
||||
paddingBottom: 20,
|
||||
},
|
||||
microphone: {
|
||||
width: 160,
|
||||
height: 160,
|
||||
},
|
||||
camera: {
|
||||
width: 120,
|
||||
height: 120,
|
||||
},
|
||||
bottomPanel: {
|
||||
paddingVertical: 10,
|
||||
},
|
||||
endButton: {
|
||||
flexDirection: 'row',
|
||||
alignItems: 'center',
|
||||
justifyContent: 'center',
|
||||
backgroundColor: 'black',
|
||||
borderRadius: 12,
|
||||
padding: 10,
|
||||
},
|
||||
endText: {
|
||||
marginLeft: 5,
|
||||
color: 'white',
|
||||
},
|
||||
});
|
||||
|
||||
export default MeetingView;
|
||||
@@ -0,0 +1,82 @@
|
||||
import {
|
||||
View,
|
||||
StyleSheet,
|
||||
Text,
|
||||
TextInput,
|
||||
Image
|
||||
} from "react-native"
|
||||
|
||||
import React, { useEffect, useState } from 'react';
|
||||
|
||||
import { useVoiceClient } from '../context/VoiceClientContext';
|
||||
|
||||
import Colors from '../theme/Colors';
|
||||
import { Images } from '../theme/Assets';
|
||||
import CustomButton from '../theme/CustomButton';
|
||||
import { SettingsManager } from '../settings/SettingsManager';
|
||||
|
||||
const styles = StyleSheet.create({
|
||||
container: {
|
||||
flex: 1,
|
||||
padding: 20,
|
||||
backgroundColor: Colors.backgroundApp,
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
},
|
||||
image: {
|
||||
width: 64,
|
||||
height: 64,
|
||||
marginBottom: 20,
|
||||
},
|
||||
header: {
|
||||
fontSize: 18,
|
||||
fontWeight: 'bold',
|
||||
marginBottom: 20,
|
||||
},
|
||||
textInput: {
|
||||
width: '100%',
|
||||
padding: 10,
|
||||
borderColor: Colors.buttonsBorder,
|
||||
backgroundColor: Colors.white,
|
||||
borderWidth: 1,
|
||||
borderRadius: 5,
|
||||
marginBottom: 10,
|
||||
},
|
||||
lastTextInput: {
|
||||
marginBottom: 20,
|
||||
},
|
||||
});
|
||||
|
||||
const PreJoinView: React.FC = () => {
|
||||
const { start } = useVoiceClient();
|
||||
|
||||
const [backendURL, setBackendURL] = useState<string>('')
|
||||
|
||||
useEffect(() => {
|
||||
const loadSettings = async () => {
|
||||
const loadedSettings = await SettingsManager.getSettings();
|
||||
setBackendURL(loadedSettings.backendURL)
|
||||
};
|
||||
loadSettings();
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<View style={styles.container}>
|
||||
<Image source={Images.dailyBot} style={styles.image} />
|
||||
<Text style={styles.header}>Connect to Pipecat.</Text>
|
||||
<TextInput
|
||||
placeholder="Server URL"
|
||||
value={backendURL}
|
||||
onChangeText={setBackendURL}
|
||||
style={[styles.textInput, styles.lastTextInput]}
|
||||
/>
|
||||
<CustomButton
|
||||
title="Connect"
|
||||
onPress={() => start(backendURL)}
|
||||
backgroundColor={Colors.backgroundCircle}
|
||||
/>
|
||||
</View>
|
||||
)
|
||||
};
|
||||
|
||||
export default PreJoinView;
|
||||
29
examples/simple-chatbot/client/react-native/tsconfig.json
Normal file
29
examples/simple-chatbot/client/react-native/tsconfig.json
Normal file
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"rootDir": ".",
|
||||
"allowUnreachableCode": false,
|
||||
"allowUnusedLabels": false,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"jsx": "react-jsx",
|
||||
"lib": [
|
||||
"ESNext"
|
||||
],
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"noEmit": true,
|
||||
"noFallthroughCasesInSwitch": true,
|
||||
"noImplicitReturns": true,
|
||||
"noImplicitUseStrict": false,
|
||||
"noStrictGenericChecks": false,
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"noUnusedLocals": false,
|
||||
"noUnusedParameters": true,
|
||||
"resolveJsonModule": true,
|
||||
"skipLibCheck": true,
|
||||
"strict": true,
|
||||
"target": "ESNext",
|
||||
"verbatimModuleSyntax": false
|
||||
},
|
||||
"extends": "expo/tsconfig.base"
|
||||
}
|
||||
4979
examples/simple-chatbot/client/react-native/yarn.lock
Normal file
4979
examples/simple-chatbot/client/react-native/yarn.lock
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user