-
-
+Voice / Text Input
+
+
+Audio Processing & STT
+
+
+Translation & Classification
+
+
+Native Actions & Lock States
+
+
+LLM & Vision
+
+
+Streaming TTS
+
+
+Conversation Storage
+
+---
## Technologies Used
-* **Kotlin:** Primary programming language.
-* **Jetpack Compose:** For building the user interface.
-* **Android Jetpack:**
- * ViewModel: For managing UI-related data in a lifecycle-conscious way.
-* **Speech Recognition & Synthesis:** Android's built-in `SpeechRecognizer` and `TextToSpeech` engines.
-* **Text Classification:** MediaPipe, utilizing a custom-trained TensorFlow Lite (TFLite) model for on-device text classification.
-* **Offline Translation:** Google ML Kit for on-device language identification and translation, enabling completely offline functionality.
-* **Networking:** OkHttp for making API calls.
-* **APIs:**
- * Groq API (for LLM, can be changed as per requirement)
- * YouTube Data API v3 (for music search)
-* **Coroutines:** For asynchronous programming.
+| Category | Technology |
+|---|---|
+| Language | Kotlin |
+| UI | Jetpack Compose, Material You |
+| Architecture | MVVM, ViewModel, Coroutines, Flow |
+| On-Device ML | MediaPipe Text Classifier, TensorFlow Lite, Sherpa-ONNX (Silero VAD, Parakeet STT, VITS TTS) |
+| Translation | Google ML Kit On-Device Translation |
+| Database | Room (encrypted via Android KeyStore + AES-256-GCM) |
+| Networking | OkHttp, Retrofit |
+| TTS | Edge TTS (WebSocket), Google Cloud TTS, Sherpa-ONNX VITS, Android TextToSpeech |
+| APIs | Groq (default LLM), YouTube Data API v3, Open-Meteo (weather) |
+| Location | GMS Fused Location Provider |
+
+---
+
+## Project Structure
+
+```
+app/src/main/java/com/app/assistant/
+│
+├── AssistActivity.kt # Entry point for system assistant integration
+├── MainActivity.kt # Main UI activity and lifecycle management
+│
+├── api/ # HTTP clients and API services
+│
+├── camera/
+│ ├── ScreenCaptureService.kt # Captures screenshots periodically
+│ └── VisualBufferManager.kt # Stores captured frames with timestamps
+│
+├── classifier/
+│ └── TextClassifierHelper.kt # Local TFLite intent classifier
+│
+├── db/
+│ ├── AppDatabase.kt # Room database setup
+│ ├── EncryptionUtil.kt # Encryption helpers (AES-GCM + KeyStore)
+│ ├── DynamicConversationRepo # Handles encrypted database operations
+│ └── *Entity.kt # Database schemas
+│
+├── llm/
+│ ├── FlexibleLlmAdapter.kt # API adapter for various LLMs
+│ ├── ModelCapabilityProber.kt # Detects model vision capability
+│ └── LlmMessage.kt # LLM data schemas
+│
+├── repository/
+│ ├── ContactsRepository.kt # Matches contact names offline
+│ ├── SettingsRepository.kt # App settings manager
+│ └── WeatherRepository.kt # Location and weather logic
+│
+├── speech/
+│ ├── AudioHygieneProcessor.kt # Audio recording and processing
+│ ├── VadIntelligenceProcessor.kt # On-device speech detection (VAD)
+│ ├── VoiceStateMachine.kt # Manages voice states (listening, speaking)
+│ └── SpeechRecognizerManager.kt # Manages speech-to-text engines
+│
+├── translation/
+│ └── TranslatorManager.kt # Manages offline translation
+│
+├── tts/
+│ ├── TtsEngineSelector.kt # Switches between TTS engines
+│ ├── EdgeTtsApiManager.kt # Microsoft Edge TTS integration
+│ ├── GoogleTtsApiManager.kt # Google Cloud TTS integration
+│ ├── OfflineTtsManager.kt # On-device TTS integration
+│ └── NativeTtsManager.kt # Default system TTS fallback
+│
+└── ui/
+ ├── screen/
+ │ ├── ChatScreen.kt # Main chat screen logic
+ │ ├── ChatLayout.kt # Layout definitions for chat
+ │ ├── ConversationItem.kt # Chat item UI elements
+ │ ├── HandsFreeBar.kt # Voice UI overlay
+ │ ├── SettingsScreen.kt # Settings layout and configuration
+ │ └── UserInputField.kt # Text input and file picker UI
+ └── theme/ # Theme, styling, and typography
+```
+
+---
## Setup
-1. **Clone the repository:**
-2. **API Keys:**
- This project requires API keys for YouTube and Groq services.
+**Prerequisites**
+- Android Studio
+- Device or emulator (Android 8.0+ / API 26+)
+- Groq API key (or another OpenAI-compatible endpoint key)
+- YouTube Data API v3 key
- **Option A: Using the `local.properties` file (Recommended for development)**
- You need to create a `local.properties` file in the root directory of your project (if it doesn't already exist) and add your keys there:
- ```properties
- YOUTUBE_API_KEY=YOUR_YOUTUBE_API_KEY
- GROQ_API_KEY=YOUR_GROQ_API_KEY
- ```
- These keys are then accessed via `BuildConfig` fields in the app.
+**Steps**
+1. Clone this repository:
+ ```bash
+ git clone https://github.com/your-username/ai-assistant-android.git
+ cd ai-assistant-android
+ ```
+2. Open the project in Android Studio and let files sync.
+3. Configure your API keys (see details in the API Configuration section).
+4. Run the app on your device.
+5. (Optional) Set the app as your default assistant under **Settings → Apps → Default Apps → Digital assistant app**.
- **Option B: Entering Keys via the App UI (Recommended for quick testing/user setup)**
- Alternatively, after installing the app, you can enter the API keys directly within the application. Navigate to the **Menu** and then to the **Settings** page within the UI, and input your `YOUTUBE_API_KEY` and `GROQ_API_KEY` there.
-3. **Ensure `local.properties` is in `.gitignore`:** The `.gitignore` file should include `local.properties` to prevent your API keys from being checked into version control.
+---
-## How to Build and Run
+## API Configuration
-1. Open the project in Android Studio.
-2. Let Android Studio sync the project and download necessary dependencies.
-3. Ensure you have an Android device or emulator connected/running.
-4. Click the "Run" button (green play icon) in Android Studio.
+Choose one of these methods to configure your API keys:
-## Notes
+**Option 1: Using `local.properties`**
+Add the keys to your `local.properties` file:
+```properties
+YOUTUBE_API_KEY=YOUR_YOUTUBE_API_KEY
+GROQ_API_KEY=YOUR_GROQ_API_KEY
+EDGE_TTS_SUBSCRIPTION_KEY=YOUR_EDGE_TTS_SUBSCRIPTION_KEY
+```
+Gradle will load these during the build. (Make sure this file is not committed to git).
-* The app requests various permissions at runtime (e.g., Record Audio, Read Contacts, Call Phone, Location) as needed for its features.
-* Bluetooth SCO is managed for better compatibility with Bluetooth headsets during speech recognition.
+**Option 2: In-App Settings**
+Open the app, go to **Settings**, and paste your keys directly into the input fields. This does not require rebuilding the app.
+
+**Customizing the LLM Endpoint**
+In the settings page, you can change the default URL to any OpenAI-compatible server (like Ollama, LM Studio, Anthropic, or Gemini). The app automatically checks if the custom endpoint supports images.
+
+---
+
+## Permissions
+
+| Permission | Required For |
+|---|---|
+| `RECORD_AUDIO` | Listening to voice input |
+| `CAMERA` | Accessing the camera for live camera sharing |
+| `READ_CONTACTS` | Searching contact names to make calls |
+| `CALL_PHONE` | Making phone calls |
+| `ACCESS_FINE_LOCATION` | Checking weather and generating navigation routes |
+| `MEDIA_PROJECTION` | Recording the screen for screen vision |
+| `BLUETOOTH` / `BLUETOOTH_CONNECT` | Supporting Bluetooth headsets |
+| `POST_NOTIFICATIONS` | Showing the screen capture status bar notification |
+
+---
+
+## Supported Features Table
+
+| Feature | Offline | Cloud | Native Android | AI / LLM |
+|---|:---:|:---:|:---:|:---:|
+| Voice activity detection (VAD) | ✅ | | | |
+| Speech-to-text (Parakeet) | ✅ | | | |
+| Speech-to-text (Cloud) | | ✅ | | |
+| Intent classification | ✅ | | | |
+| Language translation | ✅ | | | |
+| Offline TTS (VITS) | ✅ | | | |
+| Neural TTS (Edge) | | ✅ | | |
+| Encrypted storage | ✅ | | | |
+| Phone calls | | | ✅ | |
+| Alarms & reminders | | | ✅ | |
+| Google Maps navigation | | | ✅ | |
+| YouTube music playback | | ✅ | | |
+| Weather forecasts | | ✅ | | |
+| Contact matching | ✅ | | ✅ | |
+| Screen vision | | | ✅ | ✅ |
+| Live camera sharing | | | ✅ | ✅ |
+| General chat | | | | ✅ |
+| Custom LLM endpoints | | ✅ | | ✅ |
+| Image & file attachments | | | | ✅ |
+
+---
+
+## Privacy & Security
+
+Your data stays on your device unless it is sent to your selected LLM provider.
+
+**Database Encryption**
+The app generates an AES-256 encryption key inside Android's secure hardware Keystore. All messages and chat titles are encrypted before saving. Each record uses a unique initialization vector.
+
+**File Encryption**
+All saved images, documents, and screenshots are encrypted before they are written to disk.
+
+**Secure Deletion**
+When you delete a message or chat, the app deletes the database records and removes the associated files from disk.
+
+**Voice Processing**
+Your raw voice audio is never stored or sent anywhere. Voice detection is handled entirely in memory on the device, and only the transcribed text is processed.
+
+---
+
+## Known Issues
+
+- **Barge-In Interruption:** Sometimes the barge-in/interruption feature does not trigger correctly when speaking over the assistant. Work is underway to refine this.
+- **Translation Disabled:** The translation feature is currently disabled due to conflicts with streaming text output. A new approach is being developed.
+- **Settings Classification:** Settings-related commands (unlike Call, Play Song, etc., which work) do not have a separate action yet. Instead, they are classified under `OTHER` and sent directly to the LLM. A new approach is being worked on to support settings actions natively.
+
+---
+
+## Roadmap
+
+- [ ] Android XR support for smart glasses
+- [ ] Wear OS support for wrist control
+- [ ] Category for sending text messages through actions
+- [ ] Automatic summary of older conversations
+- [ ] Screen capture using Accessibility Services to avoid permission prompts
+
+---
+
+## Contributing
+
+Contributions are welcome. Please open an issue to discuss new features before making changes, or open a pull request directly for bug fixes.
+
+1. Fork the repository and create a branch.
+2. Follow the project's coding style (standard Kotlin, MVVM architecture).
+3. Submit a pull request explaining your changes.
+
+Make sure your changes do not store unencrypted user data or bypass security features.
+
+---
## License
-This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
-Copyright (c) 2025 Sourav Anand
+This project is licensed under the [MIT License](LICENSE).
+
+Copyright © 2026 Sourav Anand.
diff --git a/app/build.gradle.kts b/app/build.gradle.kts
index 208fdb2..f8ef375 100644
--- a/app/build.gradle.kts
+++ b/app/build.gradle.kts
@@ -4,7 +4,8 @@ import java.io.FileInputStream
plugins {
alias(libs.plugins.android.application)
alias(libs.plugins.jetbrains.kotlin.android)
- kotlin("plugin.serialization") version "2.0.0"
+ alias(libs.plugins.jetbrains.kotlin.serialization)
+ kotlin("kapt")
}
android {
@@ -15,8 +16,9 @@ android {
applicationId = "com.app.assistant"
minSdk = 24
targetSdk = 35
- versionCode = 1
- versionName = "1.0"
+ val runNumber = System.getenv("GITHUB_RUN_NUMBER")?.toIntOrNull() ?: 0
+ versionCode = 2 + runNumber
+ versionName = if (runNumber > 0) "2.1.$runNumber" else "2.1"
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
vectorDrawables {
@@ -39,6 +41,31 @@ android {
buildConfigField("String", "GROQ_API_KEY",
"\"${localProperties.getProperty("GROQ_API_KEY") ?: ""}\"" // Add quotes
)
+
+ buildConfigField("String", "EDGE_TTS_SUBSCRIPTION_KEY",
+ "\"${localProperties.getProperty("EDGE_TTS_SUBSCRIPTION_KEY") ?: ""}\"" // Add quotes
+ )
+ }
+
+ val keystoreFilePath = System.getenv("KEYSTORE_FILE_PATH")
+ val keystorePassword = System.getenv("KEYSTORE_PASSWORD")
+ val keystoreKeyAlias = System.getenv("KEYSTORE_KEY_ALIAS")
+ val keystoreKeyPassword = System.getenv("KEYSTORE_KEY_PASSWORD")
+
+ val hasSigningConfig = !keystoreFilePath.isNullOrEmpty() &&
+ !keystorePassword.isNullOrEmpty() &&
+ !keystoreKeyAlias.isNullOrEmpty() &&
+ !keystoreKeyPassword.isNullOrEmpty()
+
+ signingConfigs {
+ if (hasSigningConfig) {
+ create("release") {
+ storeFile = file(keystoreFilePath)
+ storePassword = keystorePassword
+ keyAlias = keystoreKeyAlias
+ keyPassword = keystoreKeyPassword
+ }
+ }
}
buildTypes {
@@ -48,8 +75,9 @@ android {
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro"
)
- // You might want to also define buildConfigFields for release
- // if you have different keys for release builds, or if local.properties is not available in CI
+ if (hasSigningConfig) {
+ signingConfig = signingConfigs.getByName("release")
+ }
}
debug {
// BuildConfigFields from defaultConfig are inherited.
@@ -69,7 +97,7 @@ android {
buildConfig = true // Ensure buildConfig is enabled
}
composeOptions {
- kotlinCompilerExtensionVersion = "1.5.1"
+ kotlinCompilerExtensionVersion = libs.versions.androidxComposeCompiler.get()
}
packaging {
resources {
@@ -88,29 +116,56 @@ dependencies {
implementation(libs.androidx.ui.graphics)
implementation(libs.androidx.ui.tooling.preview)
implementation(libs.androidx.material3)
- implementation("androidx.compose.material3:material3:1.2.1")
- implementation("androidx.compose.material3:material3-window-size-class:1.2.1")
- implementation("androidx.compose.material3:material3-adaptive-navigation-suite:1.3.0-beta02")
- implementation("com.squareup.okhttp3:okhttp:4.12.0")
- implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.6.3")
- implementation ("com.google.mlkit:translate:17.0.3")
- implementation ("org.commonmark:commonmark:0.24.0")
+ implementation(libs.androidx.material3.window.size)
+ implementation(libs.androidx.material3.adaptive.navigation.suite)
+ implementation(libs.okhttp)
+ constraints {
+ implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.6.3") {
+ because("Kotlin 1.9.0 compatibility")
+ }
+ implementation("org.jetbrains.kotlinx:kotlinx-serialization-core:1.6.3") {
+ because("Kotlin 1.9.0 compatibility")
+ }
+ }
+ implementation(libs.kotlinx.serialization.json)
+ implementation(libs.mlkit.translate)
+ implementation(libs.mlkit.barcode.scanning)
+ implementation(libs.commonmark)
//implementation ("com.github.jeziellago:compose-markdown:0.5.4")
- implementation("io.coil-kt.coil3:coil-compose:3.0.0-rc02")
- implementation("io.coil-kt.coil3:coil-network-okhttp:3.0.0-rc02")
- implementation ("org.jetbrains.kotlin:kotlin-reflect")
+ implementation(libs.coil.compose)
+ implementation(libs.coil.network.okhttp)
+ implementation(libs.androidx.room.runtime)
+ implementation(libs.androidx.room.ktx)
+ kapt(libs.androidx.room.compiler)
implementation(libs.play.services.location)
implementation(libs.androidx.ui.test.android)
- implementation("androidx.security:security-crypto:1.1.0")
+ implementation(libs.androidx.security.crypto)
implementation(libs.androidx.runtime.saveable)
testImplementation(libs.junit)
-// androidTestImplementation(libs.androidx.junit)
-// androidTestImplementation(libs.androidx.espresso.core)
-// androidTestImplementation(platform(libs.androidx.compose.bom))
-// androidTestImplementation(libs.androidx.ui.test.junit4)
+ androidTestImplementation(libs.androidx.junit)
+ androidTestImplementation(libs.androidx.espresso.core)
+ androidTestImplementation(platform(libs.androidx.compose.bom))
+ androidTestImplementation(libs.androidx.ui.test.junit4)
debugImplementation(libs.androidx.ui.tooling)
debugImplementation(libs.androidx.ui.test.manifest)
//MediaPipe library
- implementation ("com.google.mediapipe:tasks-text:0.10.29")
-}
\ No newline at end of file
+ implementation(libs.mediapipe.tasks.text)
+
+ // Sherpa ONNX Offline STT
+ implementation(files("libs/sherpa-onnx-1.13.2.aar"))
+
+ // CameraX dependencies
+ implementation(libs.androidx.camera.core)
+ implementation(libs.androidx.camera.camera2)
+ implementation(libs.androidx.camera.lifecycle)
+ implementation(libs.androidx.camera.view)
+}
+
+configurations.all {
+ resolutionStrategy {
+ force("org.jetbrains.kotlinx:kotlinx-serialization-json:1.6.3")
+ force("org.jetbrains.kotlinx:kotlinx-serialization-core:1.6.3")
+ force("org.jetbrains.kotlinx:kotlinx-serialization-core-jvm:1.6.3")
+ }
+}
diff --git a/app/libs/sherpa-onnx-1.13.2.aar b/app/libs/sherpa-onnx-1.13.2.aar
new file mode 100644
index 0000000..9686f8b
Binary files /dev/null and b/app/libs/sherpa-onnx-1.13.2.aar differ
diff --git a/app/src/main/AndroidManifest.xml b/app/src/main/AndroidManifest.xml
index 0e180da..45936ff 100644
--- a/app/src/main/AndroidManifest.xml
+++ b/app/src/main/AndroidManifest.xml
@@ -2,6 +2,10 @@
+
@@ -55,14 +63,28 @@
-
+
+
+
\ No newline at end of file
diff --git a/app/src/main/java/com/app/assistant/AssistActivity.kt b/app/src/main/java/com/app/assistant/AssistActivity.kt
index 9a4d31f..34cbd59 100644
--- a/app/src/main/java/com/app/assistant/AssistActivity.kt
+++ b/app/src/main/java/com/app/assistant/AssistActivity.kt
@@ -30,7 +30,7 @@ class AssistActivity : ComponentActivity() {
finish()
}
- private fun moveToMainActivity(){
+ private fun moveToMainActivity() {
val intent = Intent(this, MainActivity::class.java)
intent.flags = Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_CLEAR_TASK or Intent.FLAG_ACTIVITY_NO_USER_ACTION
intent.putExtra("speak", true)
@@ -47,8 +47,9 @@ class AssistActivity : ComponentActivity() {
} else {
this.window.addFlags(
WindowManager.LayoutParams.FLAG_DISMISS_KEYGUARD or
- WindowManager.LayoutParams.FLAG_SHOW_WHEN_LOCKED or
- WindowManager.LayoutParams.FLAG_TURN_SCREEN_ON)
+ WindowManager.LayoutParams.FLAG_SHOW_WHEN_LOCKED or
+ WindowManager.LayoutParams.FLAG_TURN_SCREEN_ON,
+ )
}
}
-}
\ No newline at end of file
+}
diff --git a/app/src/main/java/com/app/assistant/Classifier/TextClassifierHelper.kt b/app/src/main/java/com/app/assistant/Classifier/TextClassifierHelper.kt
deleted file mode 100644
index 4abe4c1..0000000
--- a/app/src/main/java/com/app/assistant/Classifier/TextClassifierHelper.kt
+++ /dev/null
@@ -1,71 +0,0 @@
-package com.app.assistant.Classifier
-
-import android.content.Context
-import android.os.SystemClock
-import android.util.Log
-import com.google.mediapipe.tasks.core.BaseOptions
-import com.google.mediapipe.tasks.text.textclassifier.TextClassifier
-import com.google.mediapipe.tasks.text.textclassifier.TextClassifierResult
-import java.util.concurrent.ScheduledThreadPoolExecutor
-
-class TextClassifierHelper(
- var currentModel: String = MOBILEBERT,
- val context: Context,
- val listener: TextResultsListener
-) {
- private lateinit var textClassifier: TextClassifier
- private lateinit var executor: ScheduledThreadPoolExecutor
-
- init {
- initClassifier()
- }
-
- fun initClassifier() {
- val baseOptionsBuilder = BaseOptions.builder()
- .setModelAssetPath(currentModel)
-
- try {
- val baseOptions = baseOptionsBuilder.build()
- val optionsBuilder = TextClassifier.TextClassifierOptions.builder()
- .setBaseOptions(baseOptions)
- val options = optionsBuilder.build()
- textClassifier = TextClassifier.createFromOptions(context, options)
- } catch (e: IllegalStateException) {
- listener.onError(
- "Text classifier failed to initialize. See error logs for " +
- "details"
- )
- Log.e(
- TAG, "Text classifier failed to load the task with error: " + e
- .message
- )
- }
- }
-
- // Run text classification using MediaPipe Text Classifier API
- fun classify(text: String, itemId: Long, loadingItemId: Long, speak: Boolean) {
- executor = ScheduledThreadPoolExecutor(1)
-
- executor.execute {
- // inferenceTime is the amount of time, in milliseconds, that it takes to
- // classify the input text.
- var inferenceTime = SystemClock.uptimeMillis()
-
- val results = textClassifier.classify(text)
-
- inferenceTime = SystemClock.uptimeMillis() - inferenceTime
-
- listener.onResult(results, inferenceTime, text, itemId, loadingItemId, speak)
- }
- }
-
- interface TextResultsListener {
- fun onError(error: String)
- fun onResult(results: TextClassifierResult, inferenceTime: Long, inputText: String, itemId: Long, loadingItemId: Long, speak: Boolean)
- }
-
- companion object {
- const val TAG = "TextClassifierHelper"
- const val MOBILEBERT = "model.tflite"
- }
-}
\ No newline at end of file
diff --git a/app/src/main/java/com/app/assistant/MainActivity.kt b/app/src/main/java/com/app/assistant/MainActivity.kt
index 52e5037..cb0d8fd 100644
--- a/app/src/main/java/com/app/assistant/MainActivity.kt
+++ b/app/src/main/java/com/app/assistant/MainActivity.kt
@@ -1,30 +1,506 @@
package com.app.assistant
+import android.Manifest
import android.app.KeyguardManager
+import android.content.ComponentCallbacks2
import android.content.Context
+import android.content.Intent
+import android.content.IntentSender
+import android.content.pm.PackageManager
+import android.content.res.Configuration
+import android.location.LocationManager
+import android.media.AudioDeviceInfo
+import android.media.AudioManager
import android.os.Build
import android.os.Bundle
+import android.os.Looper
+import android.os.Vibrator
+import android.os.VibrationEffect
+import android.speech.RecognitionListener
+import android.speech.RecognizerIntent
+import android.speech.SpeechRecognizer
+import android.speech.tts.TextToSpeech
+import android.speech.tts.UtteranceProgressListener
import android.util.Log
import android.view.WindowManager
+import android.widget.Toast
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.enableEdgeToEdge
import androidx.activity.viewModels
+import androidx.core.app.ActivityCompat
+import androidx.core.content.ContextCompat
+import androidx.lifecycle.Lifecycle
+import androidx.lifecycle.lifecycleScope
+import androidx.lifecycle.repeatOnLifecycle
import com.app.assistant.ui.screen.SetupUI
import com.app.assistant.viewmodel.MainViewModel
import com.app.assistant.viewmodel.MainViewModelFactory
+import com.app.assistant.viewmodel.SettingsViewModel
+import com.app.assistant.viewmodel.SettingsViewModelFactory
+import com.app.assistant.viewmodel.UIEvent
+import com.app.assistant.viewmodel.onLocationFailed
+import com.app.assistant.viewmodel.onLocationReceived
+import com.google.android.gms.common.api.ResolvableApiException
+import com.google.android.gms.location.LocationAvailability
+import com.google.android.gms.location.LocationCallback
+import com.google.android.gms.location.LocationRequest
+import com.google.android.gms.location.LocationResult
+import com.google.android.gms.location.LocationServices
+import com.google.android.gms.location.LocationSettingsRequest
+import com.google.android.gms.location.LocationSettingsResponse
+import com.google.android.gms.location.Priority
+import com.google.android.gms.location.SettingsClient
+import com.google.android.gms.tasks.Task
+import kotlinx.coroutines.delay
+import kotlinx.coroutines.launch
+import java.util.Locale
class MainActivity : ComponentActivity() {
- private val viewModel: MainViewModel by viewModels{
+ private val viewModel: MainViewModel by viewModels {
MainViewModelFactory(application, intent.getBooleanExtra("speak", false))
}
+ private val settingsViewModel: SettingsViewModel by viewModels {
+ SettingsViewModelFactory(
+ com.app.assistant.repository.SettingsRepository(application),
+ com.app.assistant.viewmodel.MainViewModelFactory.okHttpClient
+ )
+ }
+
+ private lateinit var textToSpeechManager: com.app.assistant.tts.TtsManager
+ private lateinit var speechRecognizerManager: com.app.assistant.speech.SpeechRecognizerManager
+
+ private var isScreenServiceRunning = false
+ private var isActivityStarted = false
+ private var activeCollectionJob: kotlinx.coroutines.Job? = null
+
+ private val screenCaptureLauncher = registerForActivityResult(
+ androidx.activity.result.contract.ActivityResultContracts.StartActivityForResult()
+ ) { result ->
+ if (result.resultCode == RESULT_OK && result.data != null) {
+ val serviceIntent = Intent(this, com.app.assistant.camera.ScreenCaptureService::class.java).apply {
+ action = com.app.assistant.camera.ScreenCaptureService.ACTION_START
+ putExtra(com.app.assistant.camera.ScreenCaptureService.EXTRA_RESULT_CODE, result.resultCode)
+ putExtra(com.app.assistant.camera.ScreenCaptureService.EXTRA_RESULT_DATA, result.data)
+ }
+ com.app.assistant.camera.ScreenCaptureServiceHelper.isMicMuted = viewModel.isMicMuted.value
+ com.app.assistant.camera.ScreenCaptureServiceHelper.isHandsFreeActive = viewModel.isHandsFreeModeActive.value
+ ContextCompat.startForegroundService(this, serviceIntent)
+ isScreenServiceRunning = true
+ viewModel.setScreenModeActive(true)
+ } else {
+ Toast.makeText(this, "Screen capture permission denied", Toast.LENGTH_SHORT).show()
+ isScreenServiceRunning = false
+ viewModel.setScreenModeActive(false)
+ }
+ }
+
+ private fun startScreenCaptureFlow() {
+ val mediaProjectionManager = getSystemService(Context.MEDIA_PROJECTION_SERVICE) as android.media.projection.MediaProjectionManager
+ try {
+ val intent = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
+ val config = android.media.projection.MediaProjectionConfig.createConfigForDefaultDisplay()
+ mediaProjectionManager.createScreenCaptureIntent(config)
+ } else {
+ mediaProjectionManager.createScreenCaptureIntent()
+ }
+ screenCaptureLauncher.launch(intent)
+ } catch (e: Exception) {
+ Log.e("MainActivity", "Failed to launch screen capture intent", e)
+ Toast.makeText(this, "Screen capture not supported or failed to launch", Toast.LENGTH_SHORT).show()
+ }
+ }
+
+ private fun stopScreenCaptureService() {
+ val serviceIntent = Intent(this, com.app.assistant.camera.ScreenCaptureService::class.java).apply {
+ action = com.app.assistant.camera.ScreenCaptureService.ACTION_STOP
+ }
+ startService(serviceIntent)
+ }
+
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
allowOnLockScreen()
enableEdgeToEdge()
+
+ textToSpeechManager = com.app.assistant.tts.TtsEngineSelector(
+ context = this,
+ settingsRepository = com.app.assistant.repository.SettingsRepository(application)
+ ) { isSpeaking ->
+ viewModel.setSpeaking(isSpeaking)
+ if (::speechRecognizerManager.isInitialized) {
+ speechRecognizerManager.isTtsSpeaking = isSpeaking
+ }
+ }
+ speechRecognizerManager = com.app.assistant.speech.SpeechRecognizerManager(this, lifecycleScope)
+ speechRecognizerManager.preLoadModelAsync()
+ speechRecognizerManager.shouldIgnoreUiHidden = {
+ viewModel.isScreenModeActive.value
+ }
+
+ com.app.assistant.camera.ScreenCaptureServiceHelper.onServiceStopped = {
+ runOnUiThread {
+ isScreenServiceRunning = false
+ viewModel.setScreenModeActive(false)
+ }
+ }
+ com.app.assistant.camera.ScreenCaptureServiceHelper.onToggleMuteRequested = {
+ runOnUiThread {
+ viewModel.toggleMicMute()
+ }
+ }
+
+
+ lifecycleScope.launch {
+ viewModel.isScreenModeActive.collect { active ->
+ if (active) {
+ if (!isScreenServiceRunning) {
+ startScreenCaptureFlow()
+ }
+ } else {
+ if (isScreenServiceRunning) {
+ isScreenServiceRunning = false
+ stopScreenCaptureService()
+ }
+ }
+ updateCollectionState()
+ }
+ }
+
setContent {
- SetupUI(viewModel)
+ SetupUI(
+ viewModel = viewModel,
+ settingsViewModel = settingsViewModel,
+ onToggleVisionMode = {
+ toggleVisionModeWithPermissionCheck()
+ },
+ onToggleScreenMode = {
+ toggleScreenModeWithPermissionCheck()
+ }
+ )
+ }
+ }
+
+ override fun onStart() {
+ super.onStart()
+ isActivityStarted = true
+ updateCollectionState()
+ }
+
+ override fun onStop() {
+ super.onStop()
+ isActivityStarted = false
+ updateCollectionState()
+ }
+
+ private fun updateCollectionState() {
+ val shouldCollect = isActivityStarted || viewModel.isScreenModeActive.value
+ if (shouldCollect) {
+ startActiveCollection()
+ } else {
+ stopActiveCollection()
+ }
+ }
+
+ private fun startActiveCollection() {
+ if (activeCollectionJob != null) return
+
+ Log.d("MainActivity", "Starting active flow collection (Activity started: $isActivityStarted, Screen mode active: ${viewModel.isScreenModeActive.value})")
+ activeCollectionJob = lifecycleScope.launch {
+ // uiEvent collection
+ launch {
+ viewModel.uiEvent.collect { event ->
+ handleUiEvent(event)
+ }
+ }
+
+ // isMicMuted collection
+ launch {
+ viewModel.isMicMuted.collect { muted ->
+ if (::speechRecognizerManager.isInitialized) {
+ speechRecognizerManager.setMicMuted(muted)
+ }
+ com.app.assistant.camera.ScreenCaptureServiceHelper.isMicMuted = muted
+ com.app.assistant.camera.ScreenCaptureServiceHelper.onStateChanged?.invoke()
+ }
+ }
+
+ // isHandsFreeModeActive collection
+ launch {
+ var wasHandsFreeActive = false
+ viewModel.isHandsFreeModeActive.collect { active ->
+ com.app.assistant.camera.ScreenCaptureServiceHelper.isHandsFreeActive = active
+ com.app.assistant.camera.ScreenCaptureServiceHelper.onStateChanged?.invoke()
+
+ if (active) {
+ wasHandsFreeActive = true
+ if (::speechRecognizerManager.isInitialized && !speechRecognizerManager.isListeningActive()) {
+ startSpeechRecognition(isHandsFree = true)
+ }
+ } else {
+ if (wasHandsFreeActive) {
+ wasHandsFreeActive = false
+ speechRecognizerManager.stop()
+ }
+ }
+ }
+ }
+ }
+ }
+
+ private fun stopActiveCollection() {
+ if (activeCollectionJob == null) return
+ Log.d("MainActivity", "Stopping active flow collection")
+ activeCollectionJob?.cancel()
+ activeCollectionJob = null
+ }
+
+ private fun handleUiEvent(event: UIEvent) {
+ when (event) {
+ is UIEvent.RequestPermissions -> {
+ ActivityCompat.requestPermissions(this@MainActivity, event.permissions, event.requestCode)
+ }
+
+ is UIEvent.StartIntent -> {
+ try {
+ val intent = event.intent
+ if (intent.action == Intent.ACTION_CALL) {
+ if (!isExternalAudioDeviceConnected()) {
+ intent.putExtra("android.telecom.extra.START_CALL_WITH_SPEAKERPHONE", true)
+ }
+ }
+ startActivity(intent)
+ } catch (e: Exception) {
+ Log.e("MainActivity", "Error starting intent", e)
+ }
+ }
+
+ is UIEvent.ShowToast -> {
+ Toast.makeText(this@MainActivity, event.message, Toast.LENGTH_SHORT).show()
+ }
+
+ is UIEvent.SpeakText -> {
+ if (!viewModel.isListening.value && !viewModel.isVoiceProcessing.value) {
+ textToSpeechManager.speak(event.text, event.queueMode)
+ }
+ }
+
+ is UIEvent.StopSpeaking -> {
+ textToSpeechManager.stop()
+ }
+
+ is UIEvent.StartSpeechRecognition -> {
+ startSpeechRecognition(isHandsFree = viewModel.isHandsFreeModeActive.value)
+ }
+
+ is UIEvent.StopSpeechRecognition -> {
+ speechRecognizerManager.stop()
+ }
+
+ is UIEvent.GetLocationForWeather -> {
+ getCurrentLocationForWeather(event)
+ }
+
+ is UIEvent.ResolveLocationSettings -> {
+ try {
+ event.exception.startResolutionForResult(this@MainActivity, 104)
+ } catch (sendEx: IntentSender.SendIntentException) {
+ Log.e("MainActivity", "Error showing location settings dialog: ${sendEx.message}")
+ }
+ }
+ }
+ }
+
+ private fun toggleScreenModeWithPermissionCheck() {
+ if (viewModel.isScreenModeActive.value) {
+ viewModel.setScreenModeActive(false)
+ } else {
+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
+ if (ContextCompat.checkSelfPermission(this, Manifest.permission.POST_NOTIFICATIONS) != PackageManager.PERMISSION_GRANTED) {
+ ActivityCompat.requestPermissions(this, arrayOf(Manifest.permission.POST_NOTIFICATIONS), 103)
+ return
+ }
+ }
+ startScreenCaptureFlow()
+ }
+ }
+
+ private fun startSpeechRecognition(isHandsFree: Boolean = false) {
+ if (ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
+ ActivityCompat.requestPermissions(this, arrayOf(Manifest.permission.RECORD_AUDIO), 101)
+ return
+ }
+
+ if (textToSpeechManager.isSpeaking()) {
+ textToSpeechManager.stop()
+ }
+
+ speechRecognizerManager.startListening(
+ isHandsFree = isHandsFree,
+ listener = object : com.app.assistant.speech.SpeechRecognizerManager.SpeechListener {
+ override fun onReadyForSpeech() {
+ viewModel.setMicReady(true)
+ }
+
+ override fun onAudioLevelChanged(level: Float) {
+ viewModel.setAudioAmplitude(level)
+ }
+
+ override fun onBeginningOfSpeech() {
+ viewModel.setListening(true)
+ viewModel.setVoiceProcessing(false)
+ if (textToSpeechManager.isSpeaking()) {
+ textToSpeechManager.stop()
+ }
+ }
+
+ override fun onEndOfSpeech() {
+ viewModel.setListening(false)
+ viewModel.setVoiceProcessing(true)
+ viewModel.setAudioAmplitude(0f)
+ triggerVibration()
+ }
+
+ override fun onError(errorCode: Int) {
+ viewModel.setListening(false)
+ viewModel.setVoiceProcessing(false)
+ viewModel.setMicReady(false)
+ viewModel.setAudioAmplitude(0f)
+ if (viewModel.isHandsFreeModeActive.value) {
+ lifecycleScope.launch {
+ delay(500)
+ if (viewModel.isHandsFreeModeActive.value) {
+ startSpeechRecognition(isHandsFree = true)
+ }
+ }
+ }
+ }
+
+ override fun onResults(recognizedText: String) {
+ viewModel.setListening(false)
+ viewModel.setVoiceProcessing(false)
+ viewModel.setAudioAmplitude(0f)
+ if (!viewModel.isHandsFreeModeActive.value) {
+ viewModel.setMicReady(false)
+ }
+ if (recognizedText.isNotBlank()) {
+ if (viewModel.isVisionModeActive.value || viewModel.isScreenModeActive.value) {
+ val startTimestamp = speechRecognizerManager.speechStartTimestamp
+ viewModel.onSpeechRecognizedWithVision(recognizedText, startTimestamp)
+ } else {
+ viewModel.onSpeechRecognized(recognizedText)
+ }
+ }
+ }
+
+ override fun onPartialResults(recognizedText: String) {
+ viewModel.onSpeechPartialResult(recognizedText)
+ }
+ }
+ )
+ }
+
+ private fun getCurrentLocationForWeather(event: UIEvent.GetLocationForWeather) {
+ val locationManager = getSystemService(Context.LOCATION_SERVICE) as LocationManager
+ val gpsEnabled = locationManager.isProviderEnabled(LocationManager.GPS_PROVIDER)
+
+ if (!gpsEnabled) {
+ promptEnableLocation()
+ viewModel.onLocationFailed(event.loadingItemId, event.speak, "GPS_OFF")
+ } else {
+ val fusedLocationClient = LocationServices.getFusedLocationProviderClient(this)
+ val locationRequest =
+ LocationRequest
+ .Builder(Priority.PRIORITY_HIGH_ACCURACY, 1000)
+ .setWaitForAccurateLocation(true)
+ .setMaxUpdates(1)
+ .build()
+
+ val locationCallback =
+ object : LocationCallback() {
+ override fun onLocationResult(locationResult: LocationResult) {
+ val location = locationResult.lastLocation
+ if (location != null) {
+ viewModel.onLocationReceived(
+ location.latitude,
+ location.longitude,
+ event.itemId,
+ event.loadingItemId,
+ event.speak,
+ event.categoryName,
+ event.prompt,
+ )
+ } else {
+ viewModel.onLocationFailed(event.loadingItemId, event.speak, "UNAVAILABLE")
+ }
+ fusedLocationClient.removeLocationUpdates(this)
+ }
+
+ override fun onLocationAvailability(locationAvailability: LocationAvailability) {
+ if (!locationAvailability.isLocationAvailable) {
+ viewModel.onLocationFailed(event.loadingItemId, event.speak, "SUGGEST_CITY")
+ }
+ }
+ }
+
+ val permissionGranted =
+ ActivityCompat.checkSelfPermission(
+ this,
+ Manifest.permission.ACCESS_COARSE_LOCATION,
+ ) == PackageManager.PERMISSION_GRANTED
+ if (permissionGranted) {
+ fusedLocationClient.requestLocationUpdates(locationRequest, locationCallback, Looper.getMainLooper())
+ } else {
+ viewModel.onLocationFailed(event.loadingItemId, event.speak, "PERMISSION")
+ }
+ }
+ }
+
+ private fun promptEnableLocation() {
+ val locationRequest =
+ LocationRequest
+ .Builder(100, 10000)
+ .setMinUpdateIntervalMillis(5000)
+ .build()
+
+ val builder = LocationSettingsRequest.Builder().addLocationRequest(locationRequest)
+ val settingsClient: SettingsClient = LocationServices.getSettingsClient(this)
+ val task: Task = settingsClient.checkLocationSettings(builder.build())
+
+ task.addOnFailureListener { exception ->
+ if (exception is ResolvableApiException) {
+ try {
+ exception.startResolutionForResult(this@MainActivity, 104)
+ } catch (sendEx: IntentSender.SendIntentException) {
+ Log.d("Location", "Error showing location settings dialog: ${sendEx.message}")
+ }
+ }
+ }
+ }
+
+
+
+ private fun triggerVibration() {
+ try {
+ val vibrator = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
+ val manager = getSystemService(Context.VIBRATOR_MANAGER_SERVICE) as? android.os.VibratorManager
+ manager?.defaultVibrator ?: (getSystemService(Context.VIBRATOR_SERVICE) as? Vibrator)
+ } else {
+ @Suppress("DEPRECATION")
+ getSystemService(Context.VIBRATOR_SERVICE) as? Vibrator
+ }
+
+ if (vibrator?.hasVibrator() == true) {
+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
+ vibrator.vibrate(VibrationEffect.createOneShot(100, VibrationEffect.DEFAULT_AMPLITUDE))
+ } else {
+ @Suppress("DEPRECATION")
+ vibrator.vibrate(100)
+ }
+ }
+ } catch (e: Exception) {
+ Log.e("MainActivity", "Vibration failed", e)
}
}
@@ -35,7 +511,7 @@ class MainActivity : ComponentActivity() {
} else {
window.addFlags(
WindowManager.LayoutParams.FLAG_SHOW_WHEN_LOCKED or
- WindowManager.LayoutParams.FLAG_TURN_SCREEN_ON
+ WindowManager.LayoutParams.FLAG_TURN_SCREEN_ON,
)
}
@@ -46,13 +522,87 @@ class MainActivity : ComponentActivity() {
keyguardManager.requestDismissKeyguard(this, null)
}
}
- }catch (ex: Exception){
+ } catch (ex: Exception) {
ex.message?.let { Log.d("Exception Occurred", it) }
}
}
+ private fun toggleVisionModeWithPermissionCheck() {
+ if (ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
+ ActivityCompat.requestPermissions(this, arrayOf(Manifest.permission.CAMERA), 102)
+ } else {
+ viewModel.toggleVisionMode()
+ }
+ }
+
+ override fun onRequestPermissionsResult(requestCode: Int, permissions: Array, grantResults: IntArray) {
+ super.onRequestPermissionsResult(requestCode, permissions, grantResults)
+ when (requestCode) {
+ 101 -> {
+ if (grantResults.isNotEmpty() && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
+ startSpeechRecognition()
+ } else {
+ Toast.makeText(this, "Audio permission is required for speech recognition", Toast.LENGTH_SHORT).show()
+ }
+ }
+ 102 -> {
+ if (grantResults.isNotEmpty() && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
+ viewModel.toggleVisionMode()
+ } else {
+ Toast.makeText(this, "Camera permission is required for Vision Mode", Toast.LENGTH_SHORT).show()
+ }
+ }
+ 103 -> {
+ if (grantResults.isNotEmpty() && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
+ startScreenCaptureFlow()
+ } else {
+ Toast.makeText(this, "Notification permission is required to show controls in the status bar", Toast.LENGTH_LONG).show()
+ startScreenCaptureFlow()
+ }
+ }
+ }
+ }
+
+ private fun isExternalAudioDeviceConnected(): Boolean {
+ val audioManager = getSystemService(Context.AUDIO_SERVICE) as? AudioManager ?: return false
+ val devices = audioManager.getDevices(AudioManager.GET_DEVICES_OUTPUTS)
+ for (device in devices) {
+ val type = device.type
+ if (type == AudioDeviceInfo.TYPE_BLUETOOTH_A2DP ||
+ type == AudioDeviceInfo.TYPE_BLUETOOTH_SCO ||
+ type == AudioDeviceInfo.TYPE_WIRED_HEADSET ||
+ type == AudioDeviceInfo.TYPE_WIRED_HEADPHONES ||
+ type == AudioDeviceInfo.TYPE_USB_HEADSET
+ ) {
+ return true
+ }
+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
+ if (type == AudioDeviceInfo.TYPE_BLE_HEADSET) {
+ return true
+ }
+ }
+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
+ if (type == AudioDeviceInfo.TYPE_BLE_SPEAKER ||
+ type == AudioDeviceInfo.TYPE_BLE_BROADCAST
+ ) {
+ return true
+ }
+ }
+ }
+ return false
+ }
+
override fun onDestroy() {
super.onDestroy()
- viewModel.shutdownResources()
+ com.app.assistant.camera.ScreenCaptureServiceHelper.onServiceStopped = null
+ com.app.assistant.camera.ScreenCaptureServiceHelper.onToggleMuteRequested = null
+
+ if (isFinishing) {
+ if (isScreenServiceRunning) {
+ stopScreenCaptureService()
+ }
+ }
+ textToSpeechManager.shutdown()
+ speechRecognizerManager.destroy()
}
-}
\ No newline at end of file
+}
diff --git a/app/src/main/java/com/app/assistant/camera/ScreenCaptureService.kt b/app/src/main/java/com/app/assistant/camera/ScreenCaptureService.kt
new file mode 100644
index 0000000..b276f43
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/camera/ScreenCaptureService.kt
@@ -0,0 +1,412 @@
+package com.app.assistant.camera
+
+import android.app.Notification
+import android.app.NotificationChannel
+import android.app.NotificationManager
+import android.app.Service
+import android.content.Context
+import android.content.Intent
+import android.content.pm.ServiceInfo
+import android.graphics.Bitmap
+import android.graphics.PixelFormat
+import android.hardware.display.DisplayManager
+import android.hardware.display.VirtualDisplay
+import android.media.ImageReader
+import android.media.projection.MediaProjection
+import android.media.projection.MediaProjectionManager
+import android.os.Build
+import android.os.Handler
+import android.os.HandlerThread
+import android.os.IBinder
+import android.app.PendingIntent
+import android.util.Log
+import androidx.core.app.NotificationCompat
+import com.app.assistant.R
+import android.widget.RemoteViews
+import android.util.TypedValue
+import android.view.ContextThemeWrapper
+import android.content.res.ColorStateList
+import android.graphics.Color
+import androidx.core.graphics.ColorUtils
+import java.io.ByteArrayOutputStream
+import java.nio.ByteBuffer
+
+object ScreenCaptureServiceHelper {
+ var onServiceStopped: (() -> Unit)? = null
+ var onToggleMuteRequested: (() -> Unit)? = null
+
+
+ @Volatile
+ var isMicMuted: Boolean = false
+ @Volatile
+ var isHandsFreeActive: Boolean = false
+
+ var onStateChanged: (() -> Unit)? = null
+}
+
+class ScreenCaptureService : Service() {
+
+ companion object {
+ const val ACTION_START = "com.app.assistant.action.START_SCREEN_CAPTURE"
+ const val ACTION_STOP = "com.app.assistant.action.STOP_SCREEN_CAPTURE"
+ const val ACTION_TOGGLE_MUTE = "com.app.assistant.action.TOGGLE_MUTE"
+
+ const val EXTRA_RESULT_CODE = "extra_result_code"
+ const val EXTRA_RESULT_DATA = "extra_result_data"
+
+ private const val NOTIFICATION_ID = 5001
+ private const val CHANNEL_ID = "screen_capture_channel"
+ private const val TAG = "ScreenCaptureService"
+ }
+
+ private var mediaProjection: MediaProjection? = null
+ private var virtualDisplay: VirtualDisplay? = null
+ private var imageReader: ImageReader? = null
+ private var handlerThread: HandlerThread? = null
+ private var backgroundHandler: Handler? = null
+ private var lastFrameTime = 0L
+
+ override fun onBind(intent: Intent?): IBinder? = null
+
+ override fun onCreate() {
+ super.onCreate()
+ handlerThread = HandlerThread("ScreenCaptureThread").apply {
+ start()
+ }
+ backgroundHandler = Handler(handlerThread!!.looper)
+
+ ScreenCaptureServiceHelper.onStateChanged = {
+ val manager = getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager
+ manager.notify(NOTIFICATION_ID, buildNotification())
+ }
+ }
+
+ override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
+ val action = intent?.action
+ if (action == ACTION_STOP) {
+ stopServiceInternal()
+ return START_NOT_STICKY
+ }
+
+ if (action == ACTION_TOGGLE_MUTE) {
+ ScreenCaptureServiceHelper.onToggleMuteRequested?.invoke()
+ return START_NOT_STICKY
+ }
+
+
+ if (action == ACTION_START) {
+ val resultCode = intent.getIntExtra(EXTRA_RESULT_CODE, 0)
+ val resultData = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
+ intent.getParcelableExtra(EXTRA_RESULT_DATA, Intent::class.java)
+ } else {
+ @Suppress("DEPRECATION")
+ intent.getParcelableExtra(EXTRA_RESULT_DATA)
+ }
+
+ if (resultCode != 0 && resultData != null) {
+ startForegroundNotification()
+ startScreenCapture(resultCode, resultData)
+ } else {
+ Log.e(TAG, "Invalid result code or data, stopping service")
+ stopSelf()
+ }
+ }
+
+ return START_NOT_STICKY
+ }
+
+ private fun startForegroundNotification() {
+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
+ val channel = NotificationChannel(
+ CHANNEL_ID,
+ "Screen Capture Service",
+ NotificationManager.IMPORTANCE_LOW
+ ).apply {
+ description = "Notification for Screen Share Mode"
+ }
+ val manager = getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager
+ manager.createNotificationChannel(channel)
+ }
+
+ val notification = buildNotification()
+
+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
+ startForeground(
+ NOTIFICATION_ID,
+ notification,
+ ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION or ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE
+ )
+ } else {
+ startForeground(NOTIFICATION_ID, notification)
+ }
+ }
+
+ private fun getThemeColor(attr: Int, defaultColor: Int): Int {
+ val typedValue = TypedValue()
+ val themeContext = ContextThemeWrapper(this, R.style.Theme_Assistant)
+ if (themeContext.theme.resolveAttribute(attr, typedValue, true)) {
+ return typedValue.data
+ }
+ return defaultColor
+ }
+
+ private fun buildNotification(): Notification {
+ val isMuted = ScreenCaptureServiceHelper.isMicMuted
+ val isHandsFree = ScreenCaptureServiceHelper.isHandsFreeActive
+
+ val muteText = if (isMuted) "Unmute" else "Mute"
+ val muteIcon = if (isMuted) R.drawable.ic_mic_off else R.drawable.ic_mic
+
+
+ val statusText = when {
+ isMuted -> "Microphone Muted"
+ isHandsFree -> "Hands-Free Active"
+ else -> "Capturing screen for interactions"
+ }
+
+ val flags = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
+ PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE
+ } else {
+ PendingIntent.FLAG_UPDATE_CURRENT
+ }
+
+ val stopIntent = Intent(this, ScreenCaptureService::class.java).apply {
+ action = ACTION_STOP
+ }
+ val stopPendingIntent = PendingIntent.getService(this, 1, stopIntent, flags)
+
+ val muteIntent = Intent(this, ScreenCaptureService::class.java).apply {
+ action = ACTION_TOGGLE_MUTE
+ }
+ val mutePendingIntent = PendingIntent.getService(this, 2, muteIntent, flags)
+
+
+ // Resolve wallpaper-based Material You accent or app theme accent
+ val dynamicPrimary = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
+ getColor(android.R.color.system_accent1_600)
+ } else {
+ getThemeColor(android.R.attr.colorAccent, 0xFF6650a4.toInt())
+ }
+
+ // Determine dark mode or light mode from configuration
+ val isDarkMode = (resources.configuration.uiMode and android.content.res.Configuration.UI_MODE_NIGHT_MASK) == android.content.res.Configuration.UI_MODE_NIGHT_YES
+
+ // Determine base text and accent colors depending on dark/light configuration
+ val primaryTextColor = if (isDarkMode) Color.WHITE else Color.BLACK
+ val secondaryTextColor = if (isDarkMode) 0xFFB0B0B0.toInt() else 0xFF666666.toInt()
+ val normalTint = if (isDarkMode) 0x1AFFFFFF else 0x1A000000
+ val activeTint = ColorUtils.setAlphaComponent(dynamicPrimary, if (isDarkMode) 0x66 else 0x4D) // higher opacity in dark mode for better visibility
+
+ val collapsedLayout = RemoteViews(packageName, R.layout.notification_screen_capture_collapsed).apply {
+ setTextViewText(R.id.tv_status_text, statusText)
+ setOnClickPendingIntent(R.id.btn_stop_compact, stopPendingIntent)
+
+ // Apply text and stop icon/background colors dynamically
+ setTextColor(R.id.tv_title, primaryTextColor)
+ setTextColor(R.id.tv_status_text, secondaryTextColor)
+ setInt(R.id.btn_stop_compact, "setColorFilter", primaryTextColor)
+ setColorStateList(R.id.btn_stop_compact, "setBackgroundTintList", ColorStateList.valueOf(normalTint))
+ }
+
+ val expandedLayout = RemoteViews(packageName, R.layout.notification_screen_capture_expanded).apply {
+ setTextViewText(R.id.tv_status_text, statusText)
+
+ // Apply header text colors dynamically
+ setTextColor(R.id.tv_title, primaryTextColor)
+ setTextColor(R.id.tv_status_text, secondaryTextColor)
+
+ // Setup Mute Button
+ setImageViewResource(R.id.iv_mute_icon, muteIcon)
+ setTextViewText(R.id.tv_mute_text, muteText)
+ setOnClickPendingIntent(R.id.btn_mute, mutePendingIntent)
+
+ // Set mute button text, icon, and background colors dynamically
+ setTextColor(R.id.tv_mute_text, primaryTextColor)
+ setInt(R.id.iv_mute_icon, "setColorFilter", primaryTextColor)
+ setColorStateList(
+ R.id.btn_mute,
+ "setBackgroundTintList",
+ ColorStateList.valueOf(if (isMuted) activeTint else normalTint)
+ )
+
+
+ // Setup Stop Button
+ setOnClickPendingIntent(R.id.btn_stop, stopPendingIntent)
+
+ // Set stop button text, icon, and background colors dynamically
+ setTextColor(R.id.tv_stop_text, primaryTextColor)
+ setInt(R.id.iv_stop_icon, "setColorFilter", primaryTextColor)
+ setColorStateList(R.id.btn_stop, "setBackgroundTintList", ColorStateList.valueOf(normalTint))
+ }
+
+ return NotificationCompat.Builder(this, CHANNEL_ID)
+ .setSmallIcon(R.drawable.ic_screen_share)
+ .setPriority(NotificationCompat.PRIORITY_LOW)
+ .setCategory(NotificationCompat.CATEGORY_SERVICE)
+ .setStyle(NotificationCompat.DecoratedCustomViewStyle())
+ .setCustomContentView(collapsedLayout)
+ .setCustomBigContentView(expandedLayout)
+ .build()
+ }
+
+ private fun startScreenCapture(resultCode: Int, resultData: Intent) {
+ val mpManager = getSystemService(Context.MEDIA_PROJECTION_SERVICE) as MediaProjectionManager
+ try {
+ val projection = mpManager.getMediaProjection(resultCode, resultData)
+ mediaProjection = projection
+
+ projection.registerCallback(object : MediaProjection.Callback() {
+ override fun onStop() {
+ Log.d(TAG, "MediaProjection stopped by system/user")
+ stopServiceInternal()
+ }
+ }, backgroundHandler)
+
+ setupCapturePipeline(projection)
+ } catch (e: Exception) {
+ Log.e(TAG, "Failed to start media projection capture", e)
+ stopServiceInternal()
+ }
+ }
+
+ private fun setupCapturePipeline(projection: MediaProjection) {
+ val metrics = resources.displayMetrics
+ val screenWidth = metrics.widthPixels
+ val screenHeight = metrics.heightPixels
+ val densityDpi = metrics.densityDpi
+
+ // Aspect ratio calculations scaled down to width ~720px
+ val targetWidth = 720
+ val targetHeight = (targetWidth * (screenHeight.toFloat() / screenWidth.toFloat())).toInt()
+ val targetDensityDpi = (densityDpi * (targetWidth.toFloat() / screenWidth.toFloat())).toInt()
+
+ Log.d(TAG, "Screen size: ${screenWidth}x${screenHeight}. Scaling capture to ${targetWidth}x${targetHeight} at ${targetDensityDpi}dpi")
+
+ // Max images capacity of 2. Format: RGBA_8888.
+ val reader = ImageReader.newInstance(
+ targetWidth,
+ targetHeight,
+ PixelFormat.RGBA_8888,
+ 2
+ )
+ imageReader = reader
+
+ reader.setOnImageAvailableListener({ r ->
+ var image: android.media.Image? = null
+ try {
+ image = r.acquireLatestImage()
+ if (image == null) return@setOnImageAvailableListener
+
+ val now = System.currentTimeMillis()
+ if (now - lastFrameTime < 150) {
+ image.close()
+ image = null
+ return@setOnImageAvailableListener
+ }
+ lastFrameTime = now
+
+ val width = image.width
+ val height = image.height
+ val plane = image.planes[0]
+ val buffer = plane.buffer
+ val pixelStride = plane.pixelStride
+ val rowStride = plane.rowStride
+ val rowPadding = rowStride - pixelStride * width
+
+ val bitmapWidth = width + rowPadding / pixelStride
+ val bitmap = Bitmap.createBitmap(
+ bitmapWidth,
+ height,
+ Bitmap.Config.ARGB_8888
+ )
+ buffer.position(0)
+ bitmap.copyPixelsFromBuffer(buffer)
+
+ // Close image immediately to prevent VirtualDisplay stalling
+ image.close()
+ image = null
+
+ val finalBitmap = if (rowPadding > 0) {
+ val cropped = Bitmap.createBitmap(bitmap, 0, 0, width, height)
+ bitmap.recycle()
+ cropped
+ } else {
+ bitmap
+ }
+
+ val out = ByteArrayOutputStream()
+ finalBitmap.compress(Bitmap.CompressFormat.JPEG, 80, out)
+ finalBitmap.recycle()
+
+ val jpegBytes = out.toByteArray()
+ val cachedFrame = CachedFrame(jpegBytes, now)
+ VisualBufferManager.addFrame(cachedFrame)
+ } catch (e: Exception) {
+ Log.e(TAG, "Error converting image frame", e)
+ try {
+ image?.close()
+ } catch (ex: Exception) {
+ // Ignore
+ }
+ }
+ }, backgroundHandler)
+
+ // Create VirtualDisplay using ImageReader surface
+ virtualDisplay = projection.createVirtualDisplay(
+ "ScreenCaptureDisplay",
+ targetWidth,
+ targetHeight,
+ targetDensityDpi,
+ DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
+ reader.surface,
+ null,
+ backgroundHandler
+ )
+ }
+
+ private fun stopServiceInternal() {
+ Log.d(TAG, "Stopping ScreenCaptureService and cleaning up resources")
+ try {
+ virtualDisplay?.release()
+ virtualDisplay = null
+ } catch (e: Exception) {
+ Log.e(TAG, "Error releasing virtual display", e)
+ }
+
+ try {
+ imageReader?.close()
+ imageReader = null
+ } catch (e: Exception) {
+ Log.e(TAG, "Error closing image reader", e)
+ }
+
+ try {
+ mediaProjection?.stop()
+ mediaProjection = null
+ } catch (e: Exception) {
+ Log.e(TAG, "Error stopping media projection", e)
+ }
+
+ try {
+ handlerThread?.quitSafely()
+ handlerThread = null
+ backgroundHandler = null
+ } catch (e: Exception) {
+ Log.e(TAG, "Error quitting handler thread", e)
+ }
+
+ // Clear visual buffer to prevent leaks
+ VisualBufferManager.clear()
+
+ ScreenCaptureServiceHelper.onStateChanged = null
+
+ stopForeground(true)
+ stopSelf()
+ ScreenCaptureServiceHelper.onServiceStopped?.invoke()
+ }
+
+ override fun onDestroy() {
+ super.onDestroy()
+ stopServiceInternal()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/camera/VisualBufferManager.kt b/app/src/main/java/com/app/assistant/camera/VisualBufferManager.kt
new file mode 100644
index 0000000..c811827
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/camera/VisualBufferManager.kt
@@ -0,0 +1,119 @@
+package com.app.assistant.camera
+
+import android.graphics.Bitmap
+import android.graphics.BitmapFactory
+import android.graphics.ImageFormat
+import android.graphics.Matrix
+import android.graphics.Rect
+import android.graphics.YuvImage
+import androidx.camera.core.ImageProxy
+import java.io.ByteArrayOutputStream
+import java.util.Collections
+import java.util.ArrayList
+
+data class CachedFrame(val jpegBytes: ByteArray, val timestamp: Long)
+
+object VisualBufferManager {
+ private val buffer = Collections.synchronizedList(ArrayList())
+ private const val MAX_SIZE = 15
+
+ fun addFrame(frame: CachedFrame) {
+ synchronized(buffer) {
+ buffer.add(frame)
+ if (buffer.size > MAX_SIZE) {
+ buffer.removeAt(0)
+ }
+ }
+ }
+
+ fun getOptimalFrame(speechStartTimestamp: Long): CachedFrame? {
+ synchronized(buffer) {
+ if (buffer.isEmpty()) return null
+ var optimalFrame = buffer[0]
+ var minDiff = Math.abs(optimalFrame.timestamp - speechStartTimestamp)
+ for (i in 1 until buffer.size) {
+ val frame = buffer[i]
+ val diff = Math.abs(frame.timestamp - speechStartTimestamp)
+ if (diff < minDiff) {
+ minDiff = diff
+ optimalFrame = frame
+ }
+ }
+ return optimalFrame
+ }
+ }
+
+ fun clear() {
+ buffer.clear()
+ }
+}
+
+fun imageProxyToRotatedJpeg(imageProxy: ImageProxy): ByteArray? {
+ try {
+ val width = imageProxy.width
+ val height = imageProxy.height
+
+ val yBuffer = imageProxy.planes[0].buffer
+ val uBuffer = imageProxy.planes[1].buffer
+ val vBuffer = imageProxy.planes[2].buffer
+
+ // Correct size for NV21 format: width * height * 3 / 2
+ val ySize = width * height
+ val nv21 = ByteArray(ySize + ySize / 2)
+
+ // 1. Copy Y plane, stripping row stride padding if present
+ val yRowStride = imageProxy.planes[0].rowStride
+ val yBufferPos = yBuffer.position()
+ if (yRowStride == width) {
+ yBuffer.get(nv21, 0, ySize)
+ } else {
+ for (row in 0 until height) {
+ yBuffer.position(row * yRowStride)
+ yBuffer.get(nv21, row * width, width)
+ }
+ }
+ yBuffer.position(yBufferPos)
+
+ val uRowStride = imageProxy.planes[1].rowStride
+ val uPixelStride = imageProxy.planes[1].pixelStride
+ val vRowStride = imageProxy.planes[2].rowStride
+ val vPixelStride = imageProxy.planes[2].pixelStride
+
+ // 2. Interleave U and V planes into NV21 format (V, U, V, U...)
+ var pos = ySize
+ for (row in 0 until height / 2) {
+ for (col in 0 until width / 2) {
+ val uIndex = row * uRowStride + col * uPixelStride
+ val vIndex = row * vRowStride + col * vPixelStride
+
+ if (vIndex < vBuffer.capacity() && uIndex < uBuffer.capacity()) {
+ nv21[pos++] = vBuffer.get(vIndex)
+ nv21[pos++] = uBuffer.get(uIndex)
+ }
+ }
+ }
+
+ val yuvImage = YuvImage(nv21, ImageFormat.NV21, width, height, null)
+ val out = ByteArrayOutputStream()
+ yuvImage.compressToJpeg(Rect(0, 0, width, height), 80, out)
+ val jpegBytes = out.toByteArray()
+
+ val rotation = imageProxy.imageInfo.rotationDegrees
+ if (rotation == 0) {
+ return jpegBytes
+ }
+
+ val bitmap = BitmapFactory.decodeByteArray(jpegBytes, 0, jpegBytes.size) ?: return null
+ val matrix = Matrix().apply { postRotate(rotation.toFloat()) }
+ val rotatedBitmap = Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true)
+ bitmap.recycle()
+
+ val rotatedOut = ByteArrayOutputStream()
+ rotatedBitmap.compress(Bitmap.CompressFormat.JPEG, 80, rotatedOut)
+ rotatedBitmap.recycle()
+ return rotatedOut.toByteArray()
+ } catch (e: Exception) {
+ android.util.Log.e("VisualBufferManager", "Error converting ImageProxy to rotated JPEG", e)
+ return null
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/classifier/TextClassifierHelper.kt b/app/src/main/java/com/app/assistant/classifier/TextClassifierHelper.kt
new file mode 100644
index 0000000..4ffdb3e
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/classifier/TextClassifierHelper.kt
@@ -0,0 +1,117 @@
+package com.app.assistant.classifier
+
+import android.content.Context
+import android.os.SystemClock
+import android.util.Log
+import com.google.mediapipe.tasks.core.BaseOptions
+import com.google.mediapipe.tasks.text.textclassifier.TextClassifier
+import com.google.mediapipe.tasks.text.textclassifier.TextClassifierResult
+import java.util.concurrent.ScheduledThreadPoolExecutor
+
+class TextClassifierHelper(
+ var currentModel: String = MOBILEBERT,
+ val context: Context,
+ val listener: TextResultsListener,
+) {
+ private lateinit var textClassifier: TextClassifier
+ private val executor = ScheduledThreadPoolExecutor(1)
+
+ init {
+ executor.execute {
+ initClassifier()
+ }
+ }
+
+ fun initClassifier() {
+ val baseOptionsBuilder =
+ BaseOptions
+ .builder()
+ .setModelAssetPath(currentModel)
+
+ try {
+ val baseOptions = baseOptionsBuilder.build()
+ val optionsBuilder =
+ TextClassifier.TextClassifierOptions
+ .builder()
+ .setBaseOptions(baseOptions)
+ val options = optionsBuilder.build()
+ textClassifier = TextClassifier.createFromOptions(context, options)
+ } catch (e: Exception) {
+ listener.onError(
+ "Text classifier failed to initialize. See error logs for details"
+ )
+ Log.e(
+ TAG,
+ "Text classifier failed to load the task with error: " + e.message,
+ e
+ )
+ }
+ }
+
+ // Run text classification using MediaPipe Text Classifier API
+ fun classify(
+ text: String,
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ ) {
+ executor.execute {
+ if (!::textClassifier.isInitialized) {
+ listener.onError("Text classifier not initialized", itemId, loadingItemId, speak)
+ return@execute
+ }
+
+ // inferenceTime is the amount of time, in milliseconds, that it takes to
+ // classify the input text.
+ var inferenceTime = SystemClock.uptimeMillis()
+
+ // Remove all punctuation before classifying
+ val cleanedText = text.replace(Regex("[.,?!]"), "")
+ try {
+ val results = textClassifier.classify(cleanedText)
+
+ inferenceTime = SystemClock.uptimeMillis() - inferenceTime
+
+ listener.onResult(results, inferenceTime, text, itemId, loadingItemId, speak)
+ } catch (e: Exception) {
+ Log.e(TAG, "Error running classification: ${e.message}", e)
+ listener.onError("Classification failed: ${e.message}", itemId, loadingItemId, speak)
+ }
+ }
+ }
+
+ fun shutDown() {
+ try {
+ executor.shutdown()
+ } catch (e: Exception) {
+ Log.e(TAG, "Error shutting down executor", e)
+ }
+ }
+
+ interface TextResultsListener {
+ fun onError(error: String)
+
+ fun onError(
+ error: String,
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean
+ ) {
+ onError(error)
+ }
+
+ fun onResult(
+ results: TextClassifierResult,
+ inferenceTime: Long,
+ inputText: String,
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ )
+ }
+
+ companion object {
+ const val TAG = "TextClassifierHelper"
+ const val MOBILEBERT = "model.tflite"
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/config/SpeechConfig.kt b/app/src/main/java/com/app/assistant/config/SpeechConfig.kt
new file mode 100644
index 0000000..5a00794
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/config/SpeechConfig.kt
@@ -0,0 +1,71 @@
+package com.app.assistant.config
+
+import com.app.assistant.BuildConfig
+import com.app.assistant.repository.SettingsRepository
+
+import com.app.assistant.llm.LlmProvider
+
+enum class TtsApiProvider {
+ EDGE_TTS,
+ GOOGLE_TTS
+}
+
+enum class SttApiProvider {
+ GROQ_WHISPER,
+ GOOGLE_STT
+}
+
+object SpeechConfig {
+ // Current Active Providers - change these to switch implementations and rebuild
+ var ACTIVE_TTS_PROVIDER = TtsApiProvider.EDGE_TTS
+ val ACTIVE_STT_PROVIDER = SttApiProvider.GROQ_WHISPER
+
+ // Edge TTS Configurations
+ object EdgeTts {
+ const val WSS_URL = "wss://api.msedgeservices.com/tts/cognitiveservices/websocket/v1"
+ fun getSubscriptionKey(settingsRepository: SettingsRepository): String {
+ val key = settingsRepository.getEdgeTtsSubscriptionKey()
+ return if (key.isNullOrBlank()) BuildConfig.EDGE_TTS_SUBSCRIPTION_KEY else key
+ }
+ const val VOICE = "en-GB-SoniaNeural"
+ const val OUTPUT_FORMAT = "audio-24khz-48kbitrate-mono-mp3"
+ }
+
+ // Google TTS Configurations
+ object GoogleTts {
+ const val BASE_URL = "https://texttospeech.googleapis.com/v1/text:synthesize"
+ const val VOICE = "en-US-Wavenet-D"
+ const val AUDIO_ENCODING = "MP3"
+
+ fun getApiKey(settingsRepository: SettingsRepository): String {
+ return settingsRepository.getChatApiKey() ?: ""
+ }
+ }
+
+ // Groq Whisper Configurations
+ object GroqWhisper {
+ const val BASE_URL = "https://api.groq.com/openai/v1/audio/transcriptions"
+ const val MODEL = "whisper-large-v3"
+
+ fun getApiKey(settingsRepository: SettingsRepository): String {
+ var apiKey = settingsRepository.getChatApiKey()
+ val providerStr = settingsRepository.getLlmProvider()
+ val provider = try {
+ LlmProvider.valueOf(providerStr)
+ } catch (e: Exception) {
+ LlmProvider.GROQ
+ }
+ if (apiKey.isNullOrBlank() && provider == LlmProvider.GROQ) {
+ apiKey = BuildConfig.GROQ_API_KEY
+ }
+ return apiKey ?: ""
+ }
+ }
+
+ // Google STT Configurations
+ object GoogleStt {
+ const val BASE_URL = "https://speech.googleapis.com/v1/speech:recognize"
+ const val ENCODING = "LINEAR16"
+ const val SAMPLE_RATE_HERTZ = 16000
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/db/AppDatabase.kt b/app/src/main/java/com/app/assistant/db/AppDatabase.kt
new file mode 100644
index 0000000..9465e20
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/db/AppDatabase.kt
@@ -0,0 +1,32 @@
+package com.app.assistant.db
+
+import android.content.Context
+import androidx.room.Database
+import androidx.room.Room
+import androidx.room.RoomDatabase
+import androidx.room.TypeConverters
+
+@Database(entities = [GroupEntity::class, MessageEntity::class, AttachmentEntity::class], version = 4, exportSchema = false)
+@TypeConverters(Converters::class)
+abstract class AppDatabase : RoomDatabase() {
+ abstract fun conversationDao(): ConversationDao
+
+ companion object {
+ @Volatile
+ private var INSTANCE: AppDatabase? = null
+
+ fun getDatabase(context: Context): AppDatabase {
+ return INSTANCE ?: synchronized(this) {
+ val instance = Room.databaseBuilder(
+ context.applicationContext,
+ AppDatabase::class.java,
+ "conversation.db"
+ )
+ .fallbackToDestructiveMigration()
+ .build()
+ INSTANCE = instance
+ instance
+ }
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/db/AttachmentEntity.kt b/app/src/main/java/com/app/assistant/db/AttachmentEntity.kt
new file mode 100644
index 0000000..0780a53
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/db/AttachmentEntity.kt
@@ -0,0 +1,39 @@
+package com.app.assistant.db
+
+import androidx.room.ColumnInfo
+import androidx.room.Entity
+import androidx.room.ForeignKey
+import androidx.room.Index
+import androidx.room.PrimaryKey
+
+@Entity(
+ tableName = "attachments",
+ foreignKeys = [
+ ForeignKey(
+ entity = MessageEntity::class,
+ parentColumns = ["id"],
+ childColumns = ["message_id"],
+ onDelete = ForeignKey.CASCADE
+ )
+ ],
+ indices = [Index(value = ["message_id"])]
+)
+data class AttachmentEntity(
+ @PrimaryKey(autoGenerate = true)
+ val id: Long = 0,
+
+ @ColumnInfo(name = "message_id")
+ val messageId: Long,
+
+ @ColumnInfo(name = "file_path")
+ val filePath: String,
+
+ @ColumnInfo(name = "mime_type")
+ val mimeType: String,
+
+ @ColumnInfo(name = "file_name")
+ val fileName: String,
+
+ @ColumnInfo(name = "iv")
+ val iv: String
+)
diff --git a/app/src/main/java/com/app/assistant/db/ConversationDao.kt b/app/src/main/java/com/app/assistant/db/ConversationDao.kt
new file mode 100644
index 0000000..60f4cf2
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/db/ConversationDao.kt
@@ -0,0 +1,52 @@
+package com.app.assistant.db
+
+import androidx.room.Dao
+import androidx.room.Insert
+import androidx.room.OnConflictStrategy
+import androidx.room.Query
+import androidx.room.Update
+
+@Dao
+interface ConversationDao {
+ @Insert(onConflict = OnConflictStrategy.REPLACE)
+ suspend fun insertGroup(group: GroupEntity): Long
+
+ @Query("SELECT * FROM `groups` ORDER BY group_id DESC")
+ suspend fun getAllGroups(): List
+
+ @Insert(onConflict = OnConflictStrategy.REPLACE)
+ suspend fun insertMessage(message: MessageEntity)
+
+ @Query("DELETE FROM messages WHERE id = :messageId")
+ suspend fun deleteMessageById(messageId: Long)
+
+ @Update
+ suspend fun updateMessage(message: MessageEntity)
+
+ @Query("DELETE FROM messages WHERE id IN (:ids)")
+ suspend fun deleteMessagesByIds(ids: List)
+
+ @Query("SELECT file_path FROM attachments WHERE message_id IN (SELECT id FROM messages WHERE group_id = :groupId)")
+ suspend fun getAttachmentFilePathsForGroup(groupId: Long): List
+
+ @Query("DELETE FROM attachments WHERE message_id IN (SELECT id FROM messages WHERE group_id = :groupId)")
+ suspend fun deleteAttachmentsForGroup(groupId: Long)
+
+ @Query("DELETE FROM messages WHERE group_id = :groupId")
+ suspend fun deleteMessagesForGroup(groupId: Long)
+
+ @Query("DELETE FROM `groups` WHERE group_id = :groupId")
+ suspend fun deleteGroupById(groupId: Long)
+
+ @Query("SELECT * FROM messages WHERE group_id = :groupId ORDER BY id ASC")
+ suspend fun getMessagesForGroup(groupId: Long): List
+
+ @Insert(onConflict = OnConflictStrategy.REPLACE)
+ suspend fun insertAttachment(attachment: AttachmentEntity)
+
+ @Query("SELECT * FROM attachments WHERE message_id = :messageId ORDER BY id ASC")
+ suspend fun getAttachmentsForMessage(messageId: Long): List
+
+ @Query("DELETE FROM attachments WHERE message_id = :messageId")
+ suspend fun deleteAttachmentsForMessage(messageId: Long)
+}
diff --git a/app/src/main/java/com/app/assistant/db/Converters.kt b/app/src/main/java/com/app/assistant/db/Converters.kt
new file mode 100644
index 0000000..f971691
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/db/Converters.kt
@@ -0,0 +1,16 @@
+package com.app.assistant.db
+
+import androidx.room.TypeConverter
+import java.net.URI
+
+class Converters {
+ @TypeConverter
+ fun fromString(value: String?): URI? {
+ return value?.let { URI(it) }
+ }
+
+ @TypeConverter
+ fun uriToString(uri: URI?): String? {
+ return uri?.toString()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/db/DynamicConversationDbHelper.kt b/app/src/main/java/com/app/assistant/db/DynamicConversationDbHelper.kt
deleted file mode 100644
index 9f04d63..0000000
--- a/app/src/main/java/com/app/assistant/db/DynamicConversationDbHelper.kt
+++ /dev/null
@@ -1,77 +0,0 @@
-package com.app.assistant.db
-
-import android.content.Context
-import android.database.sqlite.SQLiteDatabase
-import android.database.sqlite.SQLiteOpenHelper
-import com.app.assistant.model.Conversation
-import kotlin.reflect.full.memberProperties
-
-
-class DynamicConversationDbHelper(context: Context) : SQLiteOpenHelper(context, DATABASE_NAME, null, DATABASE_VERSION) {
-
- companion object {
- private const val DATABASE_NAME = "conversation.db"
- private const val DATABASE_VERSION = 1
-
- private const val COLUMN_IV = "iv"
-
- // Groups Table
- private const val TABLE_GROUPS = "groups"
- private const val COLUMN_GROUP_ID = "group_id"
- private const val COLUMN_TITLE = "title"
- private const val COLUMN_CREATED_AT = "created_at"
-
- // Messages Table
- private const val TABLE_MESSAGES = "messages"
- }
-
- override fun onCreate(db: SQLiteDatabase?) {
- val createGroupsTableQuery = """
- CREATE TABLE $TABLE_GROUPS (
- $COLUMN_GROUP_ID INTEGER PRIMARY KEY AUTOINCREMENT,
- $COLUMN_TITLE TEXT,
- $COLUMN_CREATED_AT INTEGER
- )
- """.trimIndent()
- db?.execSQL(createGroupsTableQuery)
-
- val createMessageTableQuery = buildCreateTableQuery()
- db?.execSQL(createMessageTableQuery)
- }
-
- override fun onUpgrade(db: SQLiteDatabase?, oldVersion: Int, newVersion: Int) {
- db?.execSQL("DROP TABLE IF EXISTS $TABLE_MESSAGES")
- db?.execSQL("DROP TABLE IF EXISTS $TABLE_GROUPS")
- onCreate(db)
- }
-
- private fun buildCreateTableQuery(): String {
- val fields = Conversation::class.memberProperties
- val columns = fields.filterNot { it.name == "isLoading" }.joinToString(", ") { field ->
- val fieldType = mapFieldTypeToSQLiteType(field.returnType.toString())
- if (field.name == "id") {
- "${field.name} INTEGER PRIMARY KEY"
- } else {
- "${field.name} $fieldType"
- }
- }
-
- return """
- CREATE TABLE $TABLE_MESSAGES (
- $columns,
- iv TEXT,
- $COLUMN_GROUP_ID INTEGER,
- FOREIGN KEY ($COLUMN_GROUP_ID) REFERENCES $TABLE_GROUPS($COLUMN_GROUP_ID)
- )
- """.trimIndent()
- }
-
- private fun mapFieldTypeToSQLiteType(kotlinType: String): String {
- return when (kotlinType) {
- "kotlin.String" -> "TEXT"
- "kotlin.Int", "kotlin.Long", "kotlin.Boolean" -> "INTEGER"
- "java.net.URI" -> "TEXT"
- else -> "TEXT"
- }
- }
-}
\ No newline at end of file
diff --git a/app/src/main/java/com/app/assistant/db/DynamicConversationRepository.kt b/app/src/main/java/com/app/assistant/db/DynamicConversationRepository.kt
index e651fd1..3452e2e 100644
--- a/app/src/main/java/com/app/assistant/db/DynamicConversationRepository.kt
+++ b/app/src/main/java/com/app/assistant/db/DynamicConversationRepository.kt
@@ -1,46 +1,32 @@
package com.app.assistant.db
-import android.content.ContentValues
import android.content.Context
-import android.database.Cursor
-import android.text.format.DateFormat
-import android.util.Log
-import androidx.compose.runtime.snapshots.SnapshotStateList
import com.app.assistant.model.Conversation
import com.app.assistant.model.Group
+import com.app.assistant.model.Attachment
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
-import java.net.URI
+import kotlinx.coroutines.sync.Mutex
+import kotlinx.coroutines.sync.withLock
import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale
-import kotlin.reflect.full.memberProperties
-import kotlin.reflect.full.primaryConstructor
-class DynamicConversationRepository(private val context: Context) {
-
- companion object {
- private const val TABLE_GROUPS = "groups"
- private const val COLUMN_GROUP_ID = "group_id"
- private const val COLUMN_TITLE = "title"
- private const val COLUMN_CREATED_AT = "created_at"
- private const val TABLE_MESSAGES = "messages"
- private const val COLUMN_MESSAGE_ID = "id"
- }
-
- private val dbHelper = DynamicConversationDbHelper(context)
+class DynamicConversationRepository(
+ context: Context,
+) {
+ private val db = AppDatabase.getDatabase(context)
+ private val dao = db.conversationDao()
var currentGroupId: Long = -1L
+ private val writeMutex = Mutex()
private suspend fun startNewChat(msg: String): Long {
- withContext(Dispatchers.IO) {
- val db = dbHelper.writableDatabase
- val contentValues = ContentValues().apply {
- put(COLUMN_TITLE, generateChatTitle(msg))
- put(COLUMN_CREATED_AT, System.nanoTime())
- }
- currentGroupId = db.insert(TABLE_GROUPS, null, contentValues)
+ return withContext(Dispatchers.IO) {
+ val title = generateChatTitle(msg)
+ val group = GroupEntity(title = title, createdAt = System.nanoTime())
+ currentGroupId = dao.insertGroup(group)
+ currentGroupId
}
- return currentGroupId
}
private fun generateChatTitle(msg: String): String {
@@ -51,151 +37,181 @@ class DynamicConversationRepository(private val context: Context) {
}
suspend fun addMessage(conversation: Conversation) {
- withContext(Dispatchers.IO) {
- if (currentGroupId == -1L) {
- currentGroupId = startNewChat(conversation.englishText.takeIf { it.isNotEmpty() } ?: conversation.translatedText)
- }
+ writeMutex.withLock {
+ withContext(Dispatchers.IO) {
+ if (currentGroupId == -1L) {
+ currentGroupId = startNewChat(conversation.text)
+ }
- val db = dbHelper.writableDatabase
- val (encryptedMap, iv) = encryptConversation(conversation)
- val values = ContentValues().apply {
- encryptedMap.forEach { (key, value) -> put(key, value) }
- put("iv", iv)
- put(COLUMN_GROUP_ID, currentGroupId)
- }
- try{
- db.insert(TABLE_MESSAGES, null, values)
- }catch (e: Exception){
- e.message?.let { Log.d("Test", it) }
+ val iv = EncryptionUtil.generateIV()
+ val (encryptedText, _) = EncryptionUtil.encrypt(conversation.text, iv)
+
+ val messageEntity = MessageEntity(
+ id = conversation.id,
+ text = encryptedText,
+ isMe = conversation.isMe,
+ category = conversation.category,
+ contentURL = conversation.contentURL,
+ navigationURI = conversation.navigationURI,
+ iv = iv,
+ groupId = currentGroupId
+ )
+ dao.insertMessage(messageEntity)
+
+ // Insert attachments
+ conversation.attachments.forEach { attachment ->
+ val attachmentEntity = AttachmentEntity(
+ id = attachment.id,
+ messageId = conversation.id,
+ filePath = attachment.filePath,
+ mimeType = attachment.mimeType,
+ fileName = attachment.fileName,
+ iv = attachment.iv
+ )
+ dao.insertAttachment(attachmentEntity)
+ }
}
}
}
suspend fun deleteMessage(id: Long) {
- withContext(Dispatchers.IO) {
- val db = dbHelper.writableDatabase
- db.delete(
- TABLE_MESSAGES,
- "id = ?",
- arrayOf(id.toString())
- )
+ writeMutex.withLock {
+ withContext(Dispatchers.IO) {
+ val attachmentEntities = dao.getAttachmentsForMessage(id)
+ attachmentEntities.forEach { att ->
+ try {
+ val file = java.io.File(att.filePath)
+ if (file.exists()) {
+ file.delete()
+ }
+ } catch (e: Exception) {
+ e.printStackTrace()
+ }
+ }
+ dao.deleteAttachmentsForMessage(id)
+ dao.deleteMessageById(id)
+ }
}
}
- suspend fun updateMessage(oldConversation: Conversation, newConversation: Conversation) {
- withContext(Dispatchers.IO) {
- val db = dbHelper.writableDatabase
- val (encryptedMap, iv) = encryptConversation(newConversation)
- val values = ContentValues().apply {
- encryptedMap.forEach { (key, value) -> put(key, value) }
- put("iv", iv)
+ suspend fun updateMessage(
+ oldConversation: Conversation,
+ newConversation: Conversation,
+ ) {
+ writeMutex.withLock {
+ withContext(Dispatchers.IO) {
+ val iv = EncryptionUtil.generateIV()
+ val (encryptedText, _) = EncryptionUtil.encrypt(newConversation.text, iv)
+
+ val messageEntity = MessageEntity(
+ id = newConversation.id,
+ text = encryptedText,
+ isMe = newConversation.isMe,
+ category = newConversation.category,
+ contentURL = newConversation.contentURL,
+ navigationURI = newConversation.navigationURI,
+ iv = iv,
+ groupId = currentGroupId
+ )
+ dao.updateMessage(messageEntity)
+
+ // Delete old attachments first and insert new ones
+ dao.deleteAttachmentsForMessage(newConversation.id)
+ newConversation.attachments.forEach { attachment ->
+ val attachmentEntity = AttachmentEntity(
+ id = attachment.id,
+ messageId = newConversation.id,
+ filePath = attachment.filePath,
+ mimeType = attachment.mimeType,
+ fileName = attachment.fileName,
+ iv = attachment.iv
+ )
+ dao.insertAttachment(attachmentEntity)
+ }
}
- db.update(
- TABLE_MESSAGES,
- values,
- "id = ?",
- arrayOf(oldConversation.id.toString())
- )
}
}
suspend fun clearMessages(conversations: List) {
- withContext(Dispatchers.IO) {
- val db = dbHelper.writableDatabase
- val ids = conversations.joinToString(",") { it.id.toString() }
- db.execSQL("DELETE FROM $TABLE_MESSAGES WHERE id IN ($ids)")
- db.execSQL("DELETE FROM $TABLE_GROUPS WHERE $COLUMN_GROUP_ID == $currentGroupId")
- currentGroupId = -1L
+ writeMutex.withLock {
+ withContext(Dispatchers.IO) {
+ conversations.forEach { conversation ->
+ val attachmentEntities = dao.getAttachmentsForMessage(conversation.id)
+ attachmentEntities.forEach { att ->
+ try {
+ val file = java.io.File(att.filePath)
+ if (file.exists()) {
+ file.delete()
+ }
+ } catch (e: Exception) {
+ e.printStackTrace()
+ }
+ }
+ dao.deleteAttachmentsForMessage(conversation.id)
+ }
+ val ids = conversations.map { it.id }
+ dao.deleteMessagesByIds(ids)
+ dao.deleteGroupById(currentGroupId)
+ currentGroupId = -1L
+ }
+ }
+ }
+
+ suspend fun deleteGroup(groupId: Long) {
+ writeMutex.withLock {
+ withContext(Dispatchers.IO) {
+ val filePaths = dao.getAttachmentFilePathsForGroup(groupId)
+ filePaths.forEach { filePath ->
+ try {
+ val file = java.io.File(filePath)
+ if (file.exists()) {
+ file.delete()
+ }
+ } catch (e: Exception) {
+ e.printStackTrace()
+ }
+ }
+ dao.deleteAttachmentsForGroup(groupId)
+ dao.deleteMessagesForGroup(groupId)
+ dao.deleteGroupById(groupId)
+ }
}
}
suspend fun loadAllGroups(): MutableList {
- val groupList = mutableListOf()
- withContext(Dispatchers.IO) {
- val db = dbHelper.readableDatabase
- val cursor: Cursor = db.query(
- TABLE_GROUPS,
- arrayOf(COLUMN_GROUP_ID, COLUMN_TITLE),
- null,
- null,
- null,
- null,
- "$COLUMN_GROUP_ID DESC" // Sort by latest created
- )
-
- while (cursor.moveToNext()) {
- val groupId = cursor.getLong(cursor.getColumnIndexOrThrow(COLUMN_GROUP_ID))
- val title = cursor.getString(cursor.getColumnIndexOrThrow(COLUMN_TITLE))
- groupList.add(Group(groupId, title))
- }
- cursor.close()
+ return withContext(Dispatchers.IO) {
+ dao.getAllGroups().map { entity ->
+ Group(groupId = entity.groupId, title = entity.title)
+ }.toMutableList()
}
- return groupList
}
-
suspend fun loadMessagesForGroup(groupId: Long): MutableList {
- val tempList = mutableListOf()
- withContext(Dispatchers.IO) {
- val db = dbHelper.readableDatabase
- val cursor: Cursor = db.query(
- TABLE_MESSAGES,
- null,
- "$COLUMN_GROUP_ID = ?",
- arrayOf(groupId.toString()),
- null,
- null,
- COLUMN_MESSAGE_ID // Sort by latest created
- )
+ return withContext(Dispatchers.IO) {
+ dao.getMessagesForGroup(groupId).map { entity ->
+ val decryptedText = EncryptionUtil.decrypt(entity.text, entity.iv)
+ val attachmentEntities = dao.getAttachmentsForMessage(entity.id)
+ val attachments = attachmentEntities.map { attEntity ->
+ Attachment(
+ id = attEntity.id,
+ filePath = attEntity.filePath,
+ mimeType = attEntity.mimeType,
+ fileName = attEntity.fileName,
+ iv = attEntity.iv
+ )
+ }
- while (cursor.moveToNext()) {
- val conversation = cursorToConversation(cursor)
- tempList.add(conversation)
- }
- cursor.close()
+ Conversation(
+ id = entity.id,
+ text = decryptedText,
+ isMe = entity.isMe,
+ isLoading = false,
+ category = entity.category,
+ contentURL = entity.contentURL,
+ navigationURI = entity.navigationURI,
+ attachments = attachments
+ )
+ }.toMutableList()
}
- return tempList
}
-
- private fun cursorToConversation(cursor: Cursor): Conversation {
- val properties = Conversation::class.memberProperties
- val fieldMap = mutableMapOf()
- val iv = cursor.getString(cursor.getColumnIndexOrThrow("iv"))
-
- properties.filter { it.name != "isLoading" }.forEach { property ->
- val columnName = property.name
- val columnIndex = cursor.getColumnIndexOrThrow(columnName)
- val value = when (property.returnType.toString()) {
- "kotlin.String" -> cursor.getString(columnIndex)//?.let { EncryptionUtil.decrypt(it, iv) }
- "kotlin.Int" -> cursor.getInt(columnIndex)
- "kotlin.Long" -> cursor.getLong(columnIndex)
- "kotlin.Boolean" -> cursor.getString(columnIndex).toBooleanStrictOrNull() ?: false
- "java.net.URI" -> URI(cursor.getString(columnIndex))
- else -> null
- }
- fieldMap[columnName] = value
- }
-
- return Conversation::class.constructors.first().callBy(
- Conversation::class.primaryConstructor!!.parameters
- .filter { it.name != "isLoading" } // Ignore "isLoading"
- .associateWith { parameter -> fieldMap[parameter.name] }
- )
- }
-
- private fun encryptConversation(conversation: Conversation): Pair, String> {
- val properties = Conversation::class.memberProperties
- val encryptedMap = mutableMapOf()
- val iv = EncryptionUtil.generateIV()
-
- properties.forEach { property ->
- val key = property.name
- val value = property.getter.call(conversation)?.toString()
- if (value != null && key!="isLoading") {
- //encryptedMap[key] = EncryptionUtil.encrypt(value, iv).first
- encryptedMap[key] = value
- }
- }
- return Pair(encryptedMap, iv)
- }
-}
\ No newline at end of file
+}
diff --git a/app/src/main/java/com/app/assistant/db/EncryptionUtil.kt b/app/src/main/java/com/app/assistant/db/EncryptionUtil.kt
index 17474cf..c305f34 100644
--- a/app/src/main/java/com/app/assistant/db/EncryptionUtil.kt
+++ b/app/src/main/java/com/app/assistant/db/EncryptionUtil.kt
@@ -1,6 +1,9 @@
package com.app.assistant.db
+import android.security.keystore.KeyGenParameterSpec
+import android.security.keystore.KeyProperties
import android.util.Base64
+import java.security.KeyStore
import java.security.SecureRandom
import javax.crypto.Cipher
import javax.crypto.KeyGenerator
@@ -14,56 +17,142 @@ object EncryptionUtil {
private const val ALGORITHM = "AES"
private const val TRANSFORMATION = "AES/GCM/NoPadding"
private const val TAG_LENGTH_BIT = 128
+ private const val KEY_ALIAS = "com.app.assistant.database_key"
+ private const val ANDROID_KEY_STORE = "AndroidKeyStore"
- // Generate a random encryption key
- private val secretKey: SecretKey = generateSecretKey()
+ // Retrieve or generate a secure key persisted in Android Keystore
+ private val secretKey: SecretKey by lazy {
+ getOrCreateSecretKey()
+ }
// Generates a new random initialization vector (IV)
fun generateIV(): String {
val secureRandom = SecureRandom()
val iv = ByteArray(IV_SIZE)
secureRandom.nextBytes(iv)
- return Base64.encodeToString(iv, Base64.DEFAULT)
+ return Base64.encodeToString(iv, Base64.DEFAULT).trim()
}
// Encrypt a string and return encrypted text and IV
- fun encrypt(input: String, ivString: String): Pair {
- val iv = Base64.decode(ivString, Base64.DEFAULT)
- val cipher = Cipher.getInstance(TRANSFORMATION)
- val spec = GCMParameterSpec(TAG_LENGTH_BIT, iv)
- cipher.init(Cipher.ENCRYPT_MODE, secretKey, spec)
+ fun encrypt(
+ input: String,
+ ivString: String,
+ ): Pair {
+ if (input.isEmpty()) return Pair("", ivString)
+ return try {
+ val iv = Base64.decode(ivString, Base64.DEFAULT)
+ val cipher = Cipher.getInstance(TRANSFORMATION)
+ val spec = GCMParameterSpec(TAG_LENGTH_BIT, iv)
+ cipher.init(Cipher.ENCRYPT_MODE, secretKey, spec)
- val encryptedBytes = cipher.doFinal(input.toByteArray())
- val encryptedText = Base64.encodeToString(encryptedBytes, Base64.DEFAULT)
- return Pair(encryptedText, ivString)
+ val encryptedBytes = cipher.doFinal(input.toByteArray(Charsets.UTF_8))
+ val encryptedText = Base64.encodeToString(encryptedBytes, Base64.DEFAULT).trim()
+ Pair(encryptedText, ivString)
+ } catch (e: Exception) {
+ // Safe fallback to original text if encryption fails (to ensure no data loss)
+ Pair(input, ivString)
+ }
}
// Decrypt an encrypted string using IV
- fun decrypt(encryptedInput: String, ivString: String): String {
- val iv = Base64.decode(ivString, Base64.DEFAULT)
- val cipher = Cipher.getInstance(TRANSFORMATION)
- val spec = GCMParameterSpec(TAG_LENGTH_BIT, iv)
- cipher.init(Cipher.DECRYPT_MODE, secretKey, spec)
+ fun decrypt(
+ encryptedInput: String,
+ ivString: String,
+ ): String {
+ if (encryptedInput.isEmpty() || ivString.isEmpty()) return encryptedInput
+ return try {
+ val iv = Base64.decode(ivString, Base64.DEFAULT)
+ val cipher = Cipher.getInstance(TRANSFORMATION)
+ val spec = GCMParameterSpec(TAG_LENGTH_BIT, iv)
+ cipher.init(Cipher.DECRYPT_MODE, secretKey, spec)
- val encryptedBytes = Base64.decode(encryptedInput, Base64.DEFAULT)
- val decryptedBytes = cipher.doFinal(encryptedBytes)
- return String(decryptedBytes)
+ val encryptedBytes = Base64.decode(encryptedInput, Base64.DEFAULT)
+ val decryptedBytes = cipher.doFinal(encryptedBytes)
+ String(decryptedBytes, Charsets.UTF_8)
+ } catch (e: Exception) {
+ // Safe fallback to raw database value if decryption fails (e.g. for existing plaintext entries)
+ encryptedInput
+ }
}
- // Generate a new secret key
- private fun generateSecretKey(): SecretKey {
+ // Encrypt a file byte array using IV
+ fun encryptFile(inputBytes: ByteArray, ivString: String): ByteArray {
+ if (inputBytes.isEmpty()) return ByteArray(0)
+ return try {
+ val iv = Base64.decode(ivString, Base64.DEFAULT)
+ val cipher = Cipher.getInstance(TRANSFORMATION)
+ val spec = GCMParameterSpec(TAG_LENGTH_BIT, iv)
+ cipher.init(Cipher.ENCRYPT_MODE, secretKey, spec)
+ cipher.doFinal(inputBytes)
+ } catch (e: Exception) {
+ inputBytes
+ }
+ }
+
+ // Decrypt a file byte array using IV
+ fun decryptFile(encryptedBytes: ByteArray, ivString: String): ByteArray {
+ if (encryptedBytes.isEmpty() || ivString.isEmpty()) return encryptedBytes
+ return try {
+ val iv = Base64.decode(ivString, Base64.DEFAULT)
+ val cipher = Cipher.getInstance(TRANSFORMATION)
+ val spec = GCMParameterSpec(TAG_LENGTH_BIT, iv)
+ cipher.init(Cipher.DECRYPT_MODE, secretKey, spec)
+ cipher.doFinal(encryptedBytes)
+ } catch (e: Exception) {
+ encryptedBytes
+ }
+ }
+
+
+ // Retrieve or create a persistent AES key in Android KeyStore
+ private fun getOrCreateSecretKey(): SecretKey {
+ return try {
+ val keyStore = KeyStore.getInstance(ANDROID_KEY_STORE).apply { load(null) }
+ if (keyStore.containsAlias(KEY_ALIAS)) {
+ val entry = keyStore.getEntry(KEY_ALIAS, null) as? KeyStore.SecretKeyEntry
+ if (entry != null) {
+ return entry.secretKey
+ }
+ }
+
+ val keyGenerator = KeyGenerator.getInstance(
+ KeyProperties.KEY_ALGORITHM_AES,
+ ANDROID_KEY_STORE
+ )
+ val spec = KeyGenParameterSpec.Builder(
+ KEY_ALIAS,
+ KeyProperties.PURPOSE_ENCRYPT or KeyProperties.PURPOSE_DECRYPT
+ )
+ .setBlockModes(KeyProperties.BLOCK_MODE_GCM)
+ .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_NONE)
+ .setKeySize(KEY_SIZE)
+ .build()
+ keyGenerator.init(spec)
+ keyGenerator.generateKey()
+ } catch (e: Exception) {
+ // In-memory key fallback if Android KeyStore is unavailable (e.g., local unit tests)
+ generateFallbackSecretKey()
+ }
+ }
+
+ // Fallback key generator
+ private fun generateFallbackSecretKey(): SecretKey {
val keyGenerator = KeyGenerator.getInstance(ALGORITHM)
keyGenerator.init(KEY_SIZE, SecureRandom())
return keyGenerator.generateKey()
}
- // Save and restore secret key as Base64 (for persistence)
+ // Save and restore secret key as Base64 (kept for helper/debugging purposes)
fun getSecretKeyAsString(): String {
- return Base64.encodeToString(secretKey.encoded, Base64.DEFAULT)
+ return try {
+ Base64.encodeToString(secretKey.encoded, Base64.DEFAULT).trim()
+ } catch (e: Exception) {
+ ""
+ }
}
fun restoreSecretKeyFromString(keyString: String): SecretKey {
val decodedKey = Base64.decode(keyString, Base64.DEFAULT)
return SecretKeySpec(decodedKey, 0, decodedKey.size, ALGORITHM)
}
-}
\ No newline at end of file
+}
diff --git a/app/src/main/java/com/app/assistant/db/GroupEntity.kt b/app/src/main/java/com/app/assistant/db/GroupEntity.kt
new file mode 100644
index 0000000..4076d9d
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/db/GroupEntity.kt
@@ -0,0 +1,18 @@
+package com.app.assistant.db
+
+import androidx.room.ColumnInfo
+import androidx.room.Entity
+import androidx.room.PrimaryKey
+
+@Entity(tableName = "groups")
+data class GroupEntity(
+ @PrimaryKey(autoGenerate = true)
+ @ColumnInfo(name = "group_id")
+ val groupId: Long = 0,
+
+ @ColumnInfo(name = "title")
+ val title: String,
+
+ @ColumnInfo(name = "created_at")
+ val createdAt: Long
+)
diff --git a/app/src/main/java/com/app/assistant/db/MessageEntity.kt b/app/src/main/java/com/app/assistant/db/MessageEntity.kt
new file mode 100644
index 0000000..1a6de02
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/db/MessageEntity.kt
@@ -0,0 +1,46 @@
+package com.app.assistant.db
+
+import androidx.room.ColumnInfo
+import androidx.room.Entity
+import androidx.room.ForeignKey
+import androidx.room.Index
+import androidx.room.PrimaryKey
+import java.net.URI
+
+@Entity(
+ tableName = "messages",
+ foreignKeys = [
+ ForeignKey(
+ entity = GroupEntity::class,
+ parentColumns = ["group_id"],
+ childColumns = ["group_id"],
+ onDelete = ForeignKey.CASCADE
+ )
+ ],
+ indices = [Index(value = ["group_id"])]
+)
+data class MessageEntity(
+ @PrimaryKey
+ val id: Long,
+
+ @ColumnInfo(name = "text")
+ val text: String,
+
+ @ColumnInfo(name = "isMe")
+ val isMe: Boolean,
+
+ @ColumnInfo(name = "category")
+ val category: String,
+
+ @ColumnInfo(name = "contentURL")
+ val contentURL: String,
+
+ @ColumnInfo(name = "navigationURI")
+ val navigationURI: URI,
+
+ @ColumnInfo(name = "iv")
+ val iv: String,
+
+ @ColumnInfo(name = "group_id")
+ val groupId: Long
+)
diff --git a/app/src/main/java/com/app/assistant/db/SyncStateList.kt b/app/src/main/java/com/app/assistant/db/SyncStateList.kt
index 799354f..91d6be7 100644
--- a/app/src/main/java/com/app/assistant/db/SyncStateList.kt
+++ b/app/src/main/java/com/app/assistant/db/SyncStateList.kt
@@ -10,13 +10,13 @@ import kotlinx.coroutines.launch
class SyncStateList(
private val repository: DynamicConversationRepository, // Repository to handle DB operations
- private val stateList: SnapshotStateList = mutableStateListOf()
+ private val scope: CoroutineScope,
+ private val stateList: SnapshotStateList = mutableStateListOf(),
) : MutableList by stateList {
-
override fun add(element: Conversation): Boolean {
val result = stateList.add(element)
- if (result && !element.isLoading) {
- CoroutineScope(Dispatchers.IO).launch {
+ if (result && !element.isLoading && !element.isStreaming) {
+ scope.launch(Dispatchers.IO) {
repository.addMessage(element)
}
}
@@ -25,17 +25,26 @@ class SyncStateList(
override fun removeAt(index: Int): Conversation {
val result = stateList.removeAt(index)
- CoroutineScope(Dispatchers.IO).launch {
+ scope.launch(Dispatchers.IO) {
repository.deleteMessage(result.id)
}
return result
}
- override fun set(index: Int, element: Conversation): Conversation {
+ override fun set(
+ index: Int,
+ element: Conversation,
+ ): Conversation {
val oldElement = stateList[index]
stateList[index] = element
- CoroutineScope(Dispatchers.IO).launch {
- repository.updateMessage(oldElement, element)
+ if (!element.isLoading && !element.isStreaming) {
+ scope.launch(Dispatchers.IO) {
+ if (oldElement.isLoading || oldElement.isStreaming) {
+ repository.addMessage(element)
+ } else {
+ repository.updateMessage(oldElement, element)
+ }
+ }
}
return oldElement
}
@@ -43,7 +52,7 @@ class SyncStateList(
fun clearAll(): Job {
val oldList = stateList.toList()
stateList.clear()
- return CoroutineScope(Dispatchers.IO).launch {
+ return scope.launch(Dispatchers.IO) {
repository.clearMessages(oldList) // Suspends until DB operations finish
}
}
diff --git a/app/src/main/java/com/app/assistant/llm/FlexibleLlmAdapter.kt b/app/src/main/java/com/app/assistant/llm/FlexibleLlmAdapter.kt
new file mode 100644
index 0000000..4c3b82f
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/llm/FlexibleLlmAdapter.kt
@@ -0,0 +1,353 @@
+package com.app.assistant.llm
+
+import android.util.Log
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import kotlinx.coroutines.flow.Flow
+import kotlinx.coroutines.flow.flow
+import kotlinx.coroutines.flow.flowOn
+import kotlinx.serialization.json.Json
+import okhttp3.MediaType.Companion.toMediaTypeOrNull
+import okhttp3.OkHttpClient
+import okhttp3.Request
+import okhttp3.RequestBody.Companion.toRequestBody
+import java.io.IOException
+import java.net.UnknownHostException
+
+class FlexibleLlmAdapter(
+ private val client: OkHttpClient,
+ private val apiKey: String,
+ private val model: String,
+ private val config: LlmConfig,
+ private val provider: LlmProvider
+) : LlmAdapter {
+ private val json = Json { ignoreUnknownKeys = true }
+
+ override suspend fun generateResponse(
+ systemContext: String,
+ messages: List
+ ): String? {
+ val url = config.url
+ .replace("{{API_KEY}}", apiKey)
+ .replace("{{MODEL}}", model)
+
+ val requestBuilder = Request.Builder().url(url)
+ for ((key, value) in config.headers) {
+ val resolvedValue = value
+ .replace("{{API_KEY}}", apiKey)
+ .replace("{{MODEL}}", model)
+ requestBuilder.addHeader(key, resolvedValue)
+ }
+
+ val requestBodyString = buildRequestBodyString(systemContext, messages, isStream = false)
+ val mediaType = config.headers["Content-Type"]?.toMediaTypeOrNull() ?: "application/json".toMediaTypeOrNull()
+ val requestBody = requestBodyString.toRequestBody(mediaType)
+
+ requestBuilder.post(requestBody)
+ val request = requestBuilder.build()
+
+ val rawResponse = withContext(Dispatchers.IO) {
+ try {
+ client.newCall(request).execute().use { response ->
+ if (!response.isSuccessful) {
+ val errorBody = response.body?.string() ?: ""
+ Log.e("FlexibleLlmAdapter", "API request failed with code ${response.code}: $errorBody")
+ if (response.code == 401) {
+ return@withContext """{"error_code": 401, "msg": "API key is missing or invalid."}"""
+ }
+ val errMsg = extractErrorMessage(errorBody) ?: "API error (HTTP ${response.code})."
+ return@withContext """{"error_code": ${response.code}, "msg": "$errMsg"}"""
+ }
+ response.body!!.string()
+ }
+ } catch (e: UnknownHostException) {
+ """{"error_code": 503, "msg": "Seems this device is offline. Maybe try checking data connection."}"""
+ } catch (e: Exception) {
+ Log.e("FlexibleLlmAdapter", "Error during LLM API call", e)
+ """{"error_code": 500, "msg": "Seems some issue in LLM server. Please try again."}"""
+ }
+ }
+
+ return extractResponseContent(rawResponse, config.responsePath)
+ }
+
+ override fun generateResponseStream(
+ systemContext: String,
+ messages: List
+ ): Flow = flow {
+ var url = config.url
+ .replace("{{API_KEY}}", apiKey)
+ .replace("{{MODEL}}", model)
+
+ if (url.contains(":generateContent")) {
+ url = url.replace(":generateContent", ":streamGenerateContent")
+ }
+
+ val requestBuilder = Request.Builder().url(url)
+ for ((key, value) in config.headers) {
+ val resolvedValue = value
+ .replace("{{API_KEY}}", apiKey)
+ .replace("{{MODEL}}", model)
+ requestBuilder.addHeader(key, resolvedValue)
+ }
+
+ val requestBodyString = buildRequestBodyString(systemContext, messages, isStream = true)
+ val mediaType = config.headers["Content-Type"]?.toMediaTypeOrNull() ?: "application/json".toMediaTypeOrNull()
+ val requestBody = requestBodyString.toRequestBody(mediaType)
+
+ requestBuilder.post(requestBody)
+ val request = requestBuilder.build()
+
+ val response = try {
+ client.newCall(request).execute()
+ } catch (e: UnknownHostException) {
+ null
+ } catch (e: Exception) {
+ null
+ }
+
+ if (response == null) {
+ emit("Seems this device is offline or LLM server is unreachable.")
+ return@flow
+ }
+
+ if (!response.isSuccessful) {
+ val errorBody = response.body?.string() ?: ""
+ Log.e("FlexibleLlmAdapter", "API streaming request failed with code ${response.code}: $errorBody")
+ val errMsg = extractErrorMessage(errorBody) ?: "API error (HTTP ${response.code})."
+ emit("Error: $errMsg")
+ response.close()
+ return@flow
+ }
+
+ val responseBody = response.body
+ if (responseBody == null) {
+ emit("Error: Empty response body")
+ response.close()
+ return@flow
+ }
+
+ responseBody.byteStream().bufferedReader().use { reader ->
+ var line: String?
+ while (reader.readLine().also { line = it } != null) {
+ val currentLine = line ?: break
+ if (currentLine.startsWith("data: ")) {
+ val dataContent = currentLine.substring(6).trim()
+ if (dataContent == "[DONE]") {
+ break
+ }
+ if (dataContent.isNotEmpty()) {
+ val chunkText = extractChunkContent(dataContent, config.responsePath)
+ if (chunkText != null) {
+ emit(chunkText)
+ }
+ }
+ } else {
+ var cleanedLine = currentLine.trim()
+ if (cleanedLine.endsWith(",")) {
+ cleanedLine = cleanedLine.substring(0, cleanedLine.length - 1).trim()
+ }
+ if (cleanedLine.startsWith("{") && cleanedLine.endsWith("}")) {
+ val chunkText = extractChunkContent(cleanedLine, config.responsePath)
+ if (chunkText != null) {
+ emit(chunkText)
+ }
+ }
+ }
+ }
+ }
+ }.flowOn(Dispatchers.IO)
+
+ private fun buildRequestBodyString(systemContext: String, messages: List, isStream: Boolean): String {
+ val hasSystemContextPlaceholder = config.requestTemplate.contains("{{SYSTEM_CONTEXT}}")
+ val messagesToSerialize = if (hasSystemContextPlaceholder) {
+ messages.filter { it.role != "system" }
+ } else {
+ messages
+ }
+
+ val serializedMessagesBuilder = StringBuilder()
+ serializedMessagesBuilder.append("[")
+ messagesToSerialize.forEachIndexed { index, msg ->
+ val roleValue = when (msg.role) {
+ "system" -> config.systemRole ?: "system"
+ "user" -> config.userRole
+ "assistant" -> config.assistantRole
+ else -> msg.role
+ }
+ val msgJson = if (msg.attachments.isNullOrEmpty()) {
+ val escapedContent = escapeJsonString(msg.content)
+ config.messageFormat
+ .replace("{{ROLE}}", roleValue)
+ .replace("{{CONTENT}}", escapedContent)
+ } else {
+ buildMultimodalMessageJson(roleValue, msg.content, msg.attachments, provider)
+ }
+
+ serializedMessagesBuilder.append(msgJson)
+ if (index < messagesToSerialize.size - 1) {
+ serializedMessagesBuilder.append(",")
+ }
+ }
+ serializedMessagesBuilder.append("]")
+ val messagesJsonArray = serializedMessagesBuilder.toString()
+
+ var requestBodyString = config.requestTemplate
+ .replace("{{MODEL}}", model)
+ .replace("{{MESSAGES}}", messagesJsonArray)
+
+ if (hasSystemContextPlaceholder) {
+ requestBodyString = requestBodyString.replace("{{SYSTEM_CONTEXT}}", escapeJsonString(systemContext))
+ }
+
+ if (isStream) {
+ if (requestBodyString.contains("\"stream\": false")) {
+ requestBodyString = requestBodyString.replace("\"stream\": false", "\"stream\": true")
+ } else if (!requestBodyString.contains("\"stream\"")) {
+ requestBodyString = requestBodyString.replaceFirst("{", "{\n \"stream\": true,\n")
+ }
+ }
+ return requestBodyString
+ }
+
+ private fun buildMultimodalMessageJson(
+ role: String,
+ content: String,
+ attachments: List,
+ provider: LlmProvider
+ ): String {
+ return when (provider) {
+ LlmProvider.GEMINI -> {
+ val parts = mutableListOf()
+ parts.add("{\"text\": \"${escapeJsonString(content)}\"}")
+ attachments.forEach { att ->
+ parts.add("{\"inlineData\": {\"mimeType\": \"${att.mimeType}\", \"data\": \"${att.base64Data}\"}}")
+ }
+ "{\"role\": \"$role\", \"parts\": [${parts.joinToString(",")}]}"
+ }
+ LlmProvider.ANTHROPIC -> {
+ val contentArray = mutableListOf()
+ contentArray.add("{\"type\": \"text\", \"text\": \"${escapeJsonString(content)}\"}")
+ attachments.forEach { att ->
+ if (att.mimeType.startsWith("image/")) {
+ contentArray.add("{\"type\": \"image\", \"source\": {\"type\": \"base64\", \"media_type\": \"${att.mimeType}\", \"data\": \"${att.base64Data}\"}}")
+ } else if (att.mimeType == "application/pdf") {
+ contentArray.add("{\"type\": \"document\", \"source\": {\"type\": \"base64\", \"media_type\": \"${att.mimeType}\", \"data\": \"${att.base64Data}\"}}")
+ }
+ }
+ "{\"role\": \"$role\", \"content\": [${contentArray.joinToString(",")}]}"
+ }
+ else -> { // OpenAI, Groq, Ollama, DeepSeek, OpenRouter, Custom
+ val contentArray = mutableListOf()
+ contentArray.add("{\"type\": \"text\", \"text\": \"${escapeJsonString(content)}\"}")
+ attachments.forEach { att ->
+ if (att.mimeType.startsWith("image/")) {
+ contentArray.add("{\"type\": \"image_url\", \"image_url\": {\"url\": \"data:${att.mimeType};base64,${att.base64Data}\"}}")
+ }
+ }
+ "{\"role\": \"$role\", \"content\": [${contentArray.joinToString(",")}]}"
+ }
+ }
+ }
+
+ private fun extractChunkContent(jsonStr: String, path: String): String? {
+ var extracted = extractValueFromJson(jsonStr, path)
+ if (extracted != null) return extracted
+
+ if (path == "choices[0].message.content") {
+ extracted = extractValueFromJson(jsonStr, "choices[0].delta.content")
+ if (extracted != null) return extracted
+ }
+
+ for (fallbackPath in listOf("choices[0].delta.content", "message.content", "candidates[0].content.parts[0].text")) {
+ extracted = extractValueFromJson(jsonStr, fallbackPath)
+ if (extracted != null) return extracted
+ }
+ return null
+ }
+
+ private fun extractErrorMessage(errorBody: String): String? {
+ return try {
+ val element = json.parseToJsonElement(errorBody)
+ if (element is kotlinx.serialization.json.JsonObject) {
+ val errorObj = element["error"]
+ if (errorObj is kotlinx.serialization.json.JsonObject) {
+ val msg = errorObj["message"]
+ if (msg is kotlinx.serialization.json.JsonPrimitive) return msg.content
+ }
+ val msg = element["message"] ?: element["error_description"] ?: element["msg"]
+ if (msg is kotlinx.serialization.json.JsonPrimitive) return msg.content
+ }
+ null
+ } catch (e: Exception) {
+ errorBody.take(150)
+ }
+ }
+
+ private fun extractResponseContent(rawResponse: String, path: String): String? {
+ try {
+ if (rawResponse.contains("error_code")) {
+ val errExtract = extractValueFromJson(rawResponse, "msg")
+ if (errExtract != null) return errExtract
+ }
+
+ val extracted = extractValueFromJson(rawResponse, path) ?: return null
+ return extracted.trim()
+ } catch (e: Exception) {
+ Log.e("FlexibleLlmAdapter", "Error extracting LLM response", e)
+ return null
+ }
+ }
+
+ private fun extractValueFromJson(jsonStr: String, path: String): String? {
+ return try {
+ val jsonElement = json.parseToJsonElement(jsonStr)
+ val tokens = path.split(Regex("[\\.\\[\\]]+")).filter { it.isNotEmpty() }
+ var currentElement = jsonElement
+ for (token in tokens) {
+ currentElement = when (currentElement) {
+ is kotlinx.serialization.json.JsonObject -> {
+ currentElement[token] ?: return null
+ }
+ is kotlinx.serialization.json.JsonArray -> {
+ val index = token.toIntOrNull() ?: return null
+ if (index in 0 until currentElement.size) {
+ currentElement[index]
+ } else {
+ return null
+ }
+ }
+ else -> return null
+ }
+ }
+ if (currentElement is kotlinx.serialization.json.JsonPrimitive) {
+ currentElement.content
+ } else {
+ currentElement.toString()
+ }
+ } catch (e: Exception) {
+ null
+ }
+ }
+
+ private fun escapeJsonString(input: String): String {
+ val builder = StringBuilder()
+ for (ch in input) {
+ when (ch) {
+ '"' -> builder.append("\\\"")
+ '\\' -> builder.append("\\\\")
+ '\n' -> builder.append("\\n")
+ '\r' -> builder.append("\\r")
+ '\t' -> builder.append("\\t")
+ else -> {
+ if (ch.code < 0x20) {
+ builder.append(String.format("\\u%04x", ch.code))
+ } else {
+ builder.append(ch)
+ }
+ }
+ }
+ }
+ return builder.toString()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/llm/LlmAdapter.kt b/app/src/main/java/com/app/assistant/llm/LlmAdapter.kt
new file mode 100644
index 0000000..ff6fd4f
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/llm/LlmAdapter.kt
@@ -0,0 +1,15 @@
+package com.app.assistant.llm
+
+import kotlinx.coroutines.flow.Flow
+
+interface LlmAdapter {
+ suspend fun generateResponse(
+ systemContext: String,
+ messages: List
+ ): String?
+
+ fun generateResponseStream(
+ systemContext: String,
+ messages: List
+ ): Flow
+}
diff --git a/app/src/main/java/com/app/assistant/llm/LlmConfig.kt b/app/src/main/java/com/app/assistant/llm/LlmConfig.kt
new file mode 100644
index 0000000..dda269c
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/llm/LlmConfig.kt
@@ -0,0 +1,13 @@
+package com.app.assistant.llm
+
+data class LlmConfig(
+ val url: String,
+ val method: String = "POST",
+ val headers: Map = emptyMap(),
+ val responsePath: String,
+ val messageFormat: String,
+ val systemRole: String? = null,
+ val userRole: String = "user",
+ val assistantRole: String = "assistant",
+ val requestTemplate: String
+)
diff --git a/app/src/main/java/com/app/assistant/llm/LlmMessage.kt b/app/src/main/java/com/app/assistant/llm/LlmMessage.kt
new file mode 100644
index 0000000..58e5362
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/llm/LlmMessage.kt
@@ -0,0 +1,17 @@
+package com.app.assistant.llm
+
+import kotlinx.serialization.Serializable
+
+@Serializable
+data class LlmAttachment(
+ val base64Data: String,
+ val mimeType: String,
+ val fileName: String
+)
+
+@Serializable
+data class LlmMessage(
+ val role: String,
+ val content: String,
+ val attachments: List? = null
+)
diff --git a/app/src/main/java/com/app/assistant/llm/LlmProvider.kt b/app/src/main/java/com/app/assistant/llm/LlmProvider.kt
new file mode 100644
index 0000000..e45dc60
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/llm/LlmProvider.kt
@@ -0,0 +1,250 @@
+package com.app.assistant.llm
+
+enum class LlmProvider(
+ val displayName: String,
+ val defaultUrl: String,
+ val defaultModel: String,
+ val suggestedModels: List,
+ val config: LlmConfig
+) {
+ GROQ(
+ displayName = "Groq",
+ defaultUrl = "https://api.groq.com/openai/v1/chat/completions",
+ defaultModel = "llama-3.3-70b-versatile",
+ suggestedModels = listOf(
+ "llama-3.3-70b-versatile",
+ "llama-3.1-8b-instant",
+ "mixtral-8x7b-32768",
+ "gemma2-9b-it"
+ ),
+ config = LlmConfig(
+ url = "https://api.groq.com/openai/v1/chat/completions",
+ headers = mapOf(
+ "Authorization" to "Bearer {{API_KEY}}",
+ "Content-Type" to "application/json"
+ ),
+ responsePath = "choices[0].message.content",
+ messageFormat = "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}",
+ systemRole = "system",
+ userRole = "user",
+ assistantRole = "assistant",
+ requestTemplate = """
+ {
+ "model": "{{MODEL}}",
+ "messages": {{MESSAGES}},
+ "temperature": 1.0,
+ "top_p": 1.0
+ }
+ """.trimIndent()
+ )
+ ),
+ OPENAI(
+ displayName = "OpenAI",
+ defaultUrl = "https://api.openai.com/v1/chat/completions",
+ defaultModel = "gpt-4o",
+ suggestedModels = listOf(
+ "gpt-4o",
+ "gpt-4o-mini",
+ "gpt-4-turbo",
+ "gpt-3.5-turbo"
+ ),
+ config = LlmConfig(
+ url = "https://api.openai.com/v1/chat/completions",
+ headers = mapOf(
+ "Authorization" to "Bearer {{API_KEY}}",
+ "Content-Type" to "application/json"
+ ),
+ responsePath = "choices[0].message.content",
+ messageFormat = "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}",
+ systemRole = "system",
+ userRole = "user",
+ assistantRole = "assistant",
+ requestTemplate = """
+ {
+ "model": "{{MODEL}}",
+ "messages": {{MESSAGES}},
+ "temperature": 0.7
+ }
+ """.trimIndent()
+ )
+ ),
+ OPEN_ROUTER(
+ displayName = "OpenRouter",
+ defaultUrl = "https://openrouter.ai/api/v1/chat/completions",
+ defaultModel = "meta-llama/llama-3-8b-instruct:free",
+ suggestedModels = listOf(
+ "meta-llama/llama-3-8b-instruct:free",
+ "google/gemini-2.0-flash-exp:free",
+ "deepseek/deepseek-chat",
+ "mistralai/mistral-7b-instruct:free"
+ ),
+ config = LlmConfig(
+ url = "https://openrouter.ai/api/v1/chat/completions",
+ headers = mapOf(
+ "Authorization" to "Bearer {{API_KEY}}",
+ "Content-Type" to "application/json"
+ ),
+ responsePath = "choices[0].message.content",
+ messageFormat = "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}",
+ systemRole = "system",
+ userRole = "user",
+ assistantRole = "assistant",
+ requestTemplate = """
+ {
+ "model": "{{MODEL}}",
+ "messages": {{MESSAGES}}
+ }
+ """.trimIndent()
+ )
+ ),
+ DEEPSEEK(
+ displayName = "DeepSeek",
+ defaultUrl = "https://api.deepseek.com/chat/completions",
+ defaultModel = "deepseek-chat",
+ suggestedModels = listOf(
+ "deepseek-chat",
+ "deepseek-coder"
+ ),
+ config = LlmConfig(
+ url = "https://api.deepseek.com/chat/completions",
+ headers = mapOf(
+ "Authorization" to "Bearer {{API_KEY}}",
+ "Content-Type" to "application/json"
+ ),
+ responsePath = "choices[0].message.content",
+ messageFormat = "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}",
+ systemRole = "system",
+ userRole = "user",
+ assistantRole = "assistant",
+ requestTemplate = """
+ {
+ "model": "{{MODEL}}",
+ "messages": {{MESSAGES}},
+ "temperature": 1.0
+ }
+ """.trimIndent()
+ )
+ ),
+ OLLAMA(
+ displayName = "Ollama (Local)",
+ defaultUrl = "http://10.0.2.2:11434/api/chat",
+ defaultModel = "llama3",
+ suggestedModels = listOf(
+ "llama3",
+ "llama3.1",
+ "mistral",
+ "gemma2",
+ "phi3"
+ ),
+ config = LlmConfig(
+ url = "http://10.0.2.2:11434/api/chat",
+ headers = mapOf(
+ "Content-Type" to "application/json"
+ ),
+ responsePath = "message.content",
+ messageFormat = "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}",
+ systemRole = "system",
+ userRole = "user",
+ assistantRole = "assistant",
+ requestTemplate = """
+ {
+ "model": "{{MODEL}}",
+ "messages": {{MESSAGES}},
+ "stream": false
+ }
+ """.trimIndent()
+ )
+ ),
+ GEMINI(
+ displayName = "Google Gemini",
+ defaultUrl = "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent",
+ defaultModel = "gemini-1.5-flash",
+ suggestedModels = listOf(
+ "gemini-2.5-flash",
+ "gemini-2.5-pro",
+ "gemini-1.5-flash",
+ "gemini-1.5-pro",
+ "gemini-1.0-pro"
+ ),
+ config = LlmConfig(
+ url = "https://generativelanguage.googleapis.com/v1beta/models/{{MODEL}}:generateContent?key={{API_KEY}}",
+ headers = mapOf(
+ "Content-Type" to "application/json"
+ ),
+ responsePath = "candidates[0].content.parts[0].text",
+ messageFormat = "{\"role\": \"{{ROLE}}\", \"parts\": [{\"text\": \"{{CONTENT}}\"}]}",
+ systemRole = null,
+ userRole = "user",
+ assistantRole = "model",
+ requestTemplate = """
+ {
+ "contents": {{MESSAGES}},
+ "systemInstruction": {
+ "parts": [
+ {
+ "text": "{{SYSTEM_CONTEXT}}"
+ }
+ ]
+ }
+ }
+ """.trimIndent()
+ )
+ ),
+ ANTHROPIC(
+ displayName = "Anthropic Claude",
+ defaultUrl = "https://api.anthropic.com/v1/messages",
+ defaultModel = "claude-3-5-sonnet-20240620",
+ suggestedModels = listOf(
+ "claude-3-5-sonnet-latest",
+ "claude-3-5-haiku-latest",
+ "claude-3-opus-latest",
+ "claude-3-5-sonnet-20240620",
+ "claude-3-5-haiku-20241022"
+ ),
+ config = LlmConfig(
+ url = "https://api.anthropic.com/v1/messages",
+ headers = mapOf(
+ "x-api-key" to "{{API_KEY}}",
+ "anthropic-version" to "2023-06-01",
+ "content-type" to "application/json"
+ ),
+ responsePath = "content[0].text",
+ messageFormat = "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}",
+ systemRole = null,
+ userRole = "user",
+ assistantRole = "assistant",
+ requestTemplate = """
+ {
+ "model": "{{MODEL}}",
+ "max_tokens": 1024,
+ "system": "{{SYSTEM_CONTEXT}}",
+ "messages": {{MESSAGES}}
+ }
+ """.trimIndent()
+ )
+ ),
+ CUSTOM(
+ displayName = "Custom Config",
+ defaultUrl = "https://api.example.com/v1/chat/completions",
+ defaultModel = "custom-model",
+ suggestedModels = emptyList(),
+ config = LlmConfig(
+ url = "https://api.example.com/v1/chat/completions",
+ headers = mapOf(
+ "Authorization" to "Bearer {{API_KEY}}",
+ "Content-Type" to "application/json"
+ ),
+ responsePath = "choices[0].message.content",
+ messageFormat = "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}",
+ systemRole = "system",
+ userRole = "user",
+ assistantRole = "assistant",
+ requestTemplate = """
+ {
+ "model": "{{MODEL}}",
+ "messages": {{MESSAGES}}
+ }
+ """.trimIndent()
+ )
+ )
+}
diff --git a/app/src/main/java/com/app/assistant/llm/ModelCapabilities.kt b/app/src/main/java/com/app/assistant/llm/ModelCapabilities.kt
new file mode 100644
index 0000000..d51c966
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/llm/ModelCapabilities.kt
@@ -0,0 +1,8 @@
+package com.app.assistant.llm
+
+data class ModelCapabilities(
+ val hasImageInput: Boolean,
+ val hasAudioInput: Boolean,
+ val hasVideoInput: Boolean,
+ val hasDocumentInput: Boolean
+)
diff --git a/app/src/main/java/com/app/assistant/llm/ModelCapabilityProber.kt b/app/src/main/java/com/app/assistant/llm/ModelCapabilityProber.kt
new file mode 100644
index 0000000..b7600c4
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/llm/ModelCapabilityProber.kt
@@ -0,0 +1,355 @@
+package com.app.assistant.llm
+
+import android.util.Log
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import okhttp3.MediaType.Companion.toMediaTypeOrNull
+import okhttp3.OkHttpClient
+import okhttp3.Request
+import okhttp3.RequestBody.Companion.toRequestBody
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.jsonObject
+import kotlinx.serialization.json.jsonArray
+import kotlinx.serialization.json.jsonPrimitive
+
+
+object ModelCapabilityProber {
+
+ fun resolveCapabilitiesOffline(provider: LlmProvider, modelName: String): ModelCapabilities {
+ val model = modelName.lowercase()
+ return when (provider) {
+ LlmProvider.GEMINI -> {
+ val isMultimodal = model.contains("gemini-1.5") || model.contains("gemini-2.0") || model.contains("gemini-pro") || model.contains("gemini-2.5")
+ ModelCapabilities(
+ hasImageInput = isMultimodal || model.contains("vision"),
+ hasAudioInput = isMultimodal,
+ hasVideoInput = isMultimodal,
+ hasDocumentInput = isMultimodal
+ )
+ }
+ LlmProvider.OPENAI -> {
+ val hasVision = model.contains("gpt-4o") || model.contains("vision") || model.contains("gpt-4-turbo")
+ val hasAudio = model.contains("gpt-4o-audio")
+ ModelCapabilities(
+ hasImageInput = hasVision,
+ hasAudioInput = hasAudio,
+ hasVideoInput = false,
+ hasDocumentInput = false
+ )
+ }
+ LlmProvider.ANTHROPIC -> {
+ val supportsVision = model.contains("claude-3")
+ ModelCapabilities(
+ hasImageInput = supportsVision,
+ hasAudioInput = false,
+ hasVideoInput = false,
+ hasDocumentInput = supportsVision
+ )
+ }
+ LlmProvider.GROQ -> {
+ ModelCapabilities(
+ hasImageInput = model.contains("vision") || model.contains("scout"),
+ hasAudioInput = false,
+ hasVideoInput = false,
+ hasDocumentInput = false
+ )
+ }
+ LlmProvider.OLLAMA -> {
+ val isMultimodal = model.contains("llava") || model.contains("bakllava") || model.contains("minicpm") || model.contains("vision")
+ ModelCapabilities(
+ hasImageInput = isMultimodal,
+ hasAudioInput = false,
+ hasVideoInput = false,
+ hasDocumentInput = false
+ )
+ }
+ LlmProvider.DEEPSEEK -> {
+ ModelCapabilities(
+ hasImageInput = model.contains("vl"),
+ hasAudioInput = false,
+ hasVideoInput = false,
+ hasDocumentInput = false
+ )
+ }
+ LlmProvider.OPEN_ROUTER -> {
+ val hasVision = model.contains("vision") || model.contains("llava") || model.contains("gpt-4o") || model.contains("claude-3") || model.contains("gemini")
+ val hasAudio = model.contains("gemini") || model.contains("audio")
+ val hasVideo = model.contains("gemini")
+ val hasDoc = model.contains("gemini") || model.contains("claude-3")
+ ModelCapabilities(
+ hasImageInput = hasVision,
+ hasAudioInput = hasAudio,
+ hasVideoInput = hasVideo,
+ hasDocumentInput = hasDoc
+ )
+ }
+ LlmProvider.CUSTOM -> {
+ ModelCapabilities(
+ hasImageInput = false,
+ hasAudioInput = false,
+ hasVideoInput = false,
+ hasDocumentInput = false
+ )
+ }
+ }
+ }
+
+ suspend fun probeTextConnection(
+ client: OkHttpClient,
+ apiKey: String,
+ model: String,
+ config: LlmConfig
+ ): Boolean = withContext(Dispatchers.IO) {
+ try {
+ val url = config.url
+ .replace("{{API_KEY}}", apiKey)
+ .replace("{{MODEL}}", model)
+
+ val requestBuilder = Request.Builder().url(url)
+ for ((key, value) in config.headers) {
+ val resolvedValue = value
+ .replace("{{API_KEY}}", apiKey)
+ .replace("{{MODEL}}", model)
+ requestBuilder.addHeader(key, resolvedValue)
+ }
+
+ val hasSystemContextPlaceholder = config.requestTemplate.contains("{{SYSTEM_CONTEXT}}")
+
+ val escapedContent = "say ok"
+ val msgJson = config.messageFormat
+ .replace("{{ROLE}}", config.userRole)
+ .replace("{{CONTENT}}", escapedContent)
+
+ val messagesJsonArray = "[$msgJson]"
+ var requestBodyString = config.requestTemplate
+ .replace("{{MODEL}}", model)
+ .replace("{{MESSAGES}}", messagesJsonArray)
+
+ if (hasSystemContextPlaceholder) {
+ requestBodyString = requestBodyString.replace("{{SYSTEM_CONTEXT}}", "test connection")
+ }
+
+ val mediaType = config.headers["Content-Type"]?.toMediaTypeOrNull() ?: "application/json".toMediaTypeOrNull()
+ val requestBody = requestBodyString.toRequestBody(mediaType)
+ requestBuilder.post(requestBody)
+
+ val request = requestBuilder.build()
+ client.newCall(request).execute().use { response ->
+ Log.d("ModelCapabilityProber", "Text test response code: ${response.code}")
+ response.isSuccessful
+ }
+ } catch (e: Exception) {
+ Log.e("ModelCapabilityProber", "Text test failed", e)
+ false
+ }
+ }
+
+ suspend fun probeVisionConnection(
+ client: OkHttpClient,
+ provider: LlmProvider,
+ apiKey: String,
+ model: String,
+ config: LlmConfig
+ ): Boolean = withContext(Dispatchers.IO) {
+ try {
+ val url = config.url
+ .replace("{{API_KEY}}", apiKey)
+ .replace("{{MODEL}}", model)
+
+ val requestBuilder = Request.Builder().url(url)
+ for ((key, value) in config.headers) {
+ val resolvedValue = value
+ .replace("{{API_KEY}}", apiKey)
+ .replace("{{MODEL}}", model)
+ requestBuilder.addHeader(key, resolvedValue)
+ }
+
+ // 2x2 transparent PNG base64
+ val base64Image = "iVBORw0KGgoAAAANSUhEUgAAAAIAAAACCAYAAABytg0kAAAAC0lEQVR4nGNgQAcAABIAAXfx+gAAAAAASUVORK5CYII="
+
+ val requestBodyString = when (provider) {
+ LlmProvider.GEMINI -> {
+ """
+ {
+ "contents": [{
+ "role": "user",
+ "parts": [
+ { "text": "is this image ok?" },
+ { "inlineData": { "mimeType": "image/png", "data": "$base64Image" } }
+ ]
+ }],
+ "generationConfig": { "maxOutputTokens": 1 }
+ }
+ """.trimIndent()
+ }
+ LlmProvider.ANTHROPIC -> {
+ """
+ {
+ "model": "$model",
+ "max_tokens": 10,
+ "messages": [{
+ "role": "user",
+ "content": [
+ { "type": "text", "text": "is this image ok?" },
+ { "type": "image", "source": { "type": "base64", "media_type": "image/png", "data": "$base64Image" } }
+ ]
+ }]
+ }
+ """.trimIndent()
+ }
+ else -> { // OpenAI-compatible schema (OpenAI, Groq, OpenRouter, Custom, Ollama, DeepSeek)
+ """
+ {
+ "model": "$model",
+ "max_tokens": 10,
+ "messages": [{
+ "role": "user",
+ "content": [
+ { "type": "text", "text": "is this image ok?" },
+ { "type": "image_url", "image_url": { "url": "data:image/png;base64,$base64Image" } }
+ ]
+ }]
+ }
+ """.trimIndent()
+ }
+ }
+
+ val mediaType = "application/json".toMediaTypeOrNull()
+ val requestBody = requestBodyString.toRequestBody(mediaType)
+ requestBuilder.post(requestBody)
+
+ val request = requestBuilder.build()
+ client.newCall(request).execute().use { response ->
+ Log.d("ModelCapabilityProber", "Vision test response code: ${response.code}")
+ response.isSuccessful
+ }
+ } catch (e: Exception) {
+ Log.e("ModelCapabilityProber", "Vision test failed", e)
+ false
+ }
+ }
+
+ suspend fun fetchAvailableModels(
+ client: OkHttpClient,
+ provider: LlmProvider,
+ apiKey: String,
+ customUrl: String = "",
+ customHeaders: String = ""
+ ): List = withContext(Dispatchers.IO) {
+ try {
+ val url = when (provider) {
+ LlmProvider.GROQ -> "https://api.groq.com/openai/v1/models"
+ LlmProvider.OPENAI -> "https://api.openai.com/v1/models"
+ LlmProvider.OPEN_ROUTER -> "https://openrouter.ai/api/v1/models"
+ LlmProvider.DEEPSEEK -> "https://api.deepseek.com/v1/models"
+ LlmProvider.OLLAMA -> {
+ val configUrl = provider.config.url
+ if (configUrl.endsWith("/api/chat")) {
+ configUrl.substringBeforeLast("/api/chat") + "/api/tags"
+ } else {
+ "http://10.0.2.2:11434/api/tags"
+ }
+ }
+ LlmProvider.GEMINI -> "https://generativelanguage.googleapis.com/v1beta/models?key=$apiKey"
+ LlmProvider.ANTHROPIC -> "https://api.anthropic.com/v1/models"
+ LlmProvider.CUSTOM -> {
+ if (customUrl.isBlank()) return@withContext emptyList()
+ if (customUrl.endsWith("/chat/completions")) {
+ customUrl.substringBeforeLast("/chat/completions") + "/models"
+ } else if (customUrl.endsWith("/v1/chat/completions")) {
+ customUrl.substringBeforeLast("/v1/chat/completions") + "/v1/models"
+ } else {
+ null
+ }
+ }
+ } ?: return@withContext emptyList()
+
+ val requestBuilder = Request.Builder().url(url).get()
+
+ when (provider) {
+ LlmProvider.GROQ, LlmProvider.OPENAI, LlmProvider.OPEN_ROUTER, LlmProvider.DEEPSEEK -> {
+ requestBuilder.addHeader("Authorization", "Bearer $apiKey")
+ requestBuilder.addHeader("Content-Type", "application/json")
+ }
+ LlmProvider.ANTHROPIC -> {
+ requestBuilder.addHeader("x-api-key", apiKey)
+ requestBuilder.addHeader("anthropic-version", "2023-06-01")
+ requestBuilder.addHeader("content-type", "application/json")
+ }
+ LlmProvider.CUSTOM -> {
+ if (customHeaders.isNotBlank()) {
+ try {
+ val parsed = parseHeadersJson(customHeaders)
+ for ((k, v) in parsed) {
+ requestBuilder.addHeader(k, v.replace("{{API_KEY}}", apiKey))
+ }
+ } catch (e: Exception) {
+ Log.e("ModelCapabilityProber", "Failed to parse custom headers", e)
+ }
+ }
+ }
+ else -> {}
+ }
+
+ val request = requestBuilder.build()
+ client.newCall(request).execute().use { response ->
+ if (!response.isSuccessful) {
+ Log.e("ModelCapabilityProber", "Model fetch response not successful: ${response.code}")
+ return@withContext emptyList()
+ }
+ val bodyString = response.body?.string() ?: return@withContext emptyList()
+
+ val element = Json.parseToJsonElement(bodyString)
+ val modelsList = mutableListOf()
+
+ when (provider) {
+ LlmProvider.OLLAMA -> {
+ val modelsArray = element.jsonObject["models"]?.jsonArray
+ modelsArray?.forEach { modelObj ->
+ modelObj.jsonObject["name"]?.jsonPrimitive?.content?.let { name ->
+ modelsList.add(name)
+ }
+ }
+ }
+ LlmProvider.GEMINI -> {
+ val modelsArray = element.jsonObject["models"]?.jsonArray
+ modelsArray?.forEach { modelObj ->
+ modelObj.jsonObject["name"]?.jsonPrimitive?.content?.let { name ->
+ val cleanedName = if (name.startsWith("models/")) name.substringAfter("models/") else name
+ modelsList.add(cleanedName)
+ }
+ }
+ }
+ else -> {
+ val dataArray = element.jsonObject["data"]?.jsonArray
+ dataArray?.forEach { modelObj ->
+ modelObj.jsonObject["id"]?.jsonPrimitive?.content?.let { id ->
+ modelsList.add(id)
+ }
+ }
+ }
+ }
+
+ modelsList.distinct().sorted()
+ }
+ } catch (e: Exception) {
+ Log.e("ModelCapabilityProber", "Failed to fetch models", e)
+ emptyList()
+ }
+ }
+
+ private fun parseHeadersJson(jsonStr: String): Map {
+ return try {
+ val element = Json.parseToJsonElement(jsonStr)
+ if (element is kotlinx.serialization.json.JsonObject) {
+ element.mapValues { (_, value) ->
+ if (value is kotlinx.serialization.json.JsonPrimitive) value.content else value.toString()
+ }
+ } else {
+ emptyMap()
+ }
+ } catch (e: Exception) {
+ emptyMap()
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/model/Attachment.kt b/app/src/main/java/com/app/assistant/model/Attachment.kt
new file mode 100644
index 0000000..33d2bc0
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/model/Attachment.kt
@@ -0,0 +1,12 @@
+package com.app.assistant.model
+
+import com.app.assistant.util.IdGenerator
+import java.io.Serializable
+
+data class Attachment(
+ val id: Long = IdGenerator.nextId(),
+ val filePath: String,
+ val mimeType: String,
+ val fileName: String,
+ val iv: String
+) : Serializable
diff --git a/app/src/main/java/com/app/assistant/model/Contact.kt b/app/src/main/java/com/app/assistant/model/Contact.kt
index 528f44a..e916933 100644
--- a/app/src/main/java/com/app/assistant/model/Contact.kt
+++ b/app/src/main/java/com/app/assistant/model/Contact.kt
@@ -1,3 +1,7 @@
package com.app.assistant.model
-data class Contact(val id: String ,val name: String, val phoneNumber: String)
\ No newline at end of file
+data class Contact(
+ val id: String,
+ val name: String,
+ val phoneNumber: String,
+)
diff --git a/app/src/main/java/com/app/assistant/model/Conversation.kt b/app/src/main/java/com/app/assistant/model/Conversation.kt
index b9aa12e..38dfdfa 100644
--- a/app/src/main/java/com/app/assistant/model/Conversation.kt
+++ b/app/src/main/java/com/app/assistant/model/Conversation.kt
@@ -1,5 +1,43 @@
package com.app.assistant.model
import com.app.assistant.util.Category
+import com.app.assistant.util.IdGenerator
import java.net.URI
-data class Conversation(var id: Long = System.currentTimeMillis() * 1_000_000 + (System.nanoTime() % 1_000_000), var englishText: String, var translatedText: String, val isMe: Boolean, val isLoading: Boolean = false, var category: String = Category.OTHER.name, val contentURL: String ="", val navigationURI: URI=URI("") )
\ No newline at end of file
+data class Conversation(
+ var id: Long = IdGenerator.nextId(),
+ var text: String,
+ val isMe: Boolean,
+ val isLoading: Boolean = false,
+ var category: String = Category.OTHER.name,
+ val contentURL: String = "",
+ val navigationURI: URI = URI(""),
+ val isStreaming: Boolean = false,
+ val attachments: List = emptyList(),
+) {
+ fun getThinkingProcess(): String? {
+ val start = text.indexOf("")
+ val end = text.indexOf(" ")
+ if (start == -1) return null
+ return if (end == -1) {
+ text.substring(start + 7)
+ } else {
+ text.substring(start + 7, end)
+ }
+ }
+
+ fun getActualAnswer(): String {
+ val end = text.indexOf("")
+ val mainText = if (end == -1) {
+ if (text.contains("")) "" else text
+ } else {
+ text.substring(end + 8)
+ }
+ if (isMe) {
+ val fileIndex = mainText.indexOf("\n\n[Attached File:")
+ if (fileIndex != -1) {
+ return mainText.substring(0, fileIndex)
+ }
+ }
+ return mainText
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/model/DeviceAction.kt b/app/src/main/java/com/app/assistant/model/DeviceAction.kt
new file mode 100644
index 0000000..67fc72a
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/model/DeviceAction.kt
@@ -0,0 +1,15 @@
+package com.app.assistant.model
+
+sealed class DeviceAction {
+ data class MakeCall(val phoneNumber: String) : DeviceAction()
+ data class NavigateTo(val location: String) : DeviceAction()
+ data class PlaySong(val videoId: String) : DeviceAction()
+ data class SearchSong(val query: String) : DeviceAction()
+ data class SetAlarm(
+ val message: String,
+ val hour: Int,
+ val minutes: Int,
+ val repeatDays: List,
+ val isReminder: Boolean
+ ) : DeviceAction()
+}
diff --git a/app/src/main/java/com/app/assistant/model/Group.kt b/app/src/main/java/com/app/assistant/model/Group.kt
index 59cd45f..0376804 100644
--- a/app/src/main/java/com/app/assistant/model/Group.kt
+++ b/app/src/main/java/com/app/assistant/model/Group.kt
@@ -1,3 +1,6 @@
package com.app.assistant.model
-data class Group(val groupId: Long, val title: String)
\ No newline at end of file
+data class Group(
+ val groupId: Long,
+ val title: String,
+)
diff --git a/app/src/main/java/com/app/assistant/repository/ContactsRepository.kt b/app/src/main/java/com/app/assistant/repository/ContactsRepository.kt
new file mode 100644
index 0000000..1e7d127
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/repository/ContactsRepository.kt
@@ -0,0 +1,113 @@
+package com.app.assistant.repository
+
+import android.annotation.SuppressLint
+import android.content.Context
+import android.provider.ContactsContract
+import com.app.assistant.model.Contact
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import kotlin.math.max
+
+class ContactsRepository(
+ private val context: Context,
+) {
+ @SuppressLint("Range")
+ suspend fun searchContact(prompt: String): Contact? {
+ val contentResolver = context.contentResolver
+ val contactList = mutableListOf()
+
+ withContext(Dispatchers.IO) {
+ val cursor =
+ contentResolver.query(
+ ContactsContract.Data.CONTENT_URI,
+ arrayOf(
+ ContactsContract.Data.CONTACT_ID,
+ ContactsContract.Data.DISPLAY_NAME,
+ ContactsContract.CommonDataKinds.Phone.NUMBER,
+ ),
+ "${ContactsContract.Data.MIMETYPE} =? AND ${ContactsContract.Data.CONTACT_ID} IS NOT NULL",
+ arrayOf(ContactsContract.CommonDataKinds.Phone.CONTENT_ITEM_TYPE),
+ null,
+ )
+
+ cursor?.use { contactCursor ->
+ while (contactCursor.moveToNext()) {
+ val contactId = contactCursor.getString(contactCursor.getColumnIndex(ContactsContract.Data.CONTACT_ID))
+ val name = contactCursor.getString(contactCursor.getColumnIndex(ContactsContract.Data.DISPLAY_NAME))
+ val phoneNumber = contactCursor.getString(contactCursor.getColumnIndex(ContactsContract.CommonDataKinds.Phone.NUMBER))
+ contactList.add(Contact(contactId, name, phoneNumber))
+ }
+ }
+ }
+
+ val contactName = cleanAndExtractName(prompt)
+ return findBestMatch(contactName, contactList)
+ }
+
+ private fun cleanAndExtractName(command: String): String {
+ val stopWords = setOf("can", "you", "make", "call", "dial", "number", "to", "please", "the", "a", "on")
+ val words = command.lowercase().trim().split(" ")
+ return words.filter { !stopWords.contains(it) }.joinToString(" ")
+ }
+
+ private fun findBestMatch(
+ inputString: String,
+ nameList: List,
+ similarityThreshold: Double = 0.7,
+ ): Contact? {
+ if (inputString.isEmpty()) {
+ return null
+ }
+ val inputLower = inputString.lowercase().trim()
+ var exactMatch: Contact? = null
+ var bestSubstringMatch: Contact? = null
+ var bestFuzzyMatch: Contact? = null
+ var bestFuzzyScore = 0.0
+
+ for (contact in nameList) {
+ val nameLower = contact.name.lowercase().trim()
+
+ // Check for exact match
+ if (inputLower == nameLower) {
+ return contact
+ }
+
+ // Check for exact match within tokens
+ val tokens = nameLower.split(" ")
+ for (token in tokens) {
+ when {
+ inputLower == token -> {
+ exactMatch = contact
+ }
+
+ token.contains(inputLower) -> {
+ bestSubstringMatch = contact
+ }
+
+ else -> {
+ val similarityScore = stringSimilarity(inputLower, token)
+ if (similarityScore > bestFuzzyScore) {
+ bestFuzzyScore = similarityScore
+ bestFuzzyMatch = contact
+ }
+ }
+ }
+ }
+ }
+
+ return when {
+ exactMatch != null -> exactMatch
+ bestSubstringMatch != null -> bestSubstringMatch
+ bestFuzzyScore >= similarityThreshold -> bestFuzzyMatch
+ else -> null
+ }
+ }
+
+ private fun stringSimilarity(
+ a: String,
+ b: String,
+ ): Double {
+ val matcher = a.commonPrefixWith(b).length.toDouble() / max(a.length, b.length).toDouble()
+ return matcher
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/repository/SettingsRepository.kt b/app/src/main/java/com/app/assistant/repository/SettingsRepository.kt
new file mode 100644
index 0000000..b406d9b
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/repository/SettingsRepository.kt
@@ -0,0 +1,184 @@
+package com.app.assistant.repository
+
+import android.content.Context
+import android.content.SharedPreferences
+import androidx.security.crypto.EncryptedSharedPreferences
+import androidx.security.crypto.MasterKeys
+
+class SettingsRepository(
+ val context: Context,
+) {
+ private val sharedPreferences: SharedPreferences =
+ context.getSharedPreferences("app_prefs", Context.MODE_PRIVATE)
+
+ private val securedPreferences: SharedPreferences =
+ EncryptedSharedPreferences.create(
+ "secure_prefs",
+ MasterKeys.getOrCreate(MasterKeys.AES256_GCM_SPEC),
+ context.applicationContext,
+ EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
+ EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM,
+ )
+
+ fun getIsTranslationEnabled(): Boolean = sharedPreferences.getBoolean("is_translation_enabled", false)
+
+ fun setTranslationEnabled(enabled: Boolean) {
+ sharedPreferences.edit().putBoolean("is_translation_enabled", enabled).apply()
+ }
+
+ fun getActiveLanguageCode(): String = sharedPreferences.getString("active_language_code", "en") ?: "en"
+
+ fun setActiveLanguageCode(languageCode: String) {
+ sharedPreferences.edit().putString("active_language_code", languageCode).apply()
+ }
+
+ fun saveKeys(
+ youtubeApiKey: String,
+ chatApiKey: String,
+ edgeTtsSubscriptionKey: String = "",
+ ) {
+ with(securedPreferences.edit()) {
+ if (youtubeApiKey.isNotBlank()) {
+ putString("youtube_api_key", youtubeApiKey)
+ }
+ if (chatApiKey.isNotBlank()) {
+ putString("chat_api_key", chatApiKey)
+ }
+ if (edgeTtsSubscriptionKey.isNotBlank()) {
+ putString("edge_tts_subscription_key", edgeTtsSubscriptionKey)
+ }
+ apply()
+ }
+ }
+
+ fun getYoutubeApiKey(): String? = securedPreferences.getString("youtube_api_key", null)
+
+ fun getChatApiKey(): String? = securedPreferences.getString("chat_api_key", null)
+
+ fun getEdgeTtsSubscriptionKey(): String? = securedPreferences.getString("edge_tts_subscription_key", null)
+
+ fun getLlmProvider(): String = sharedPreferences.getString("llm_provider", "GROQ") ?: "GROQ"
+
+ fun setLlmProvider(provider: String) {
+ sharedPreferences.edit().putString("llm_provider", provider).apply()
+ }
+
+ fun getLlmModel(): String = sharedPreferences.getString("llm_model", "llama-3.3-70b-versatile") ?: "llama-3.3-70b-versatile"
+
+ fun setLlmModel(model: String) {
+ sharedPreferences.edit().putString("llm_model", model).apply()
+ }
+
+ fun getLlmCustomUrl(): String = sharedPreferences.getString("llm_custom_url", "https://api.example.com/v1/chat/completions") ?: "https://api.example.com/v1/chat/completions"
+
+ fun setLlmCustomUrl(url: String) {
+ sharedPreferences.edit().putString("llm_custom_url", url).apply()
+ }
+
+ fun getLlmCustomHeaders(): String = sharedPreferences.getString("llm_custom_headers", "{\"Authorization\": \"Bearer {{API_KEY}}\", \"Content-Type\": \"application/json\"}") ?: "{\"Authorization\": \"Bearer {{API_KEY}}\", \"Content-Type\": \"application/json\"}"
+
+ fun setLlmCustomHeaders(headers: String) {
+ sharedPreferences.edit().putString("llm_custom_headers", headers).apply()
+ }
+
+ fun getLlmCustomResponsePath(): String = sharedPreferences.getString("llm_custom_response_path", "choices[0].message.content") ?: "choices[0].message.content"
+
+ fun setLlmCustomResponsePath(path: String) {
+ sharedPreferences.edit().putString("llm_custom_response_path", path).apply()
+ }
+
+ fun getLlmCustomRequestTemplate(): String = sharedPreferences.getString("llm_custom_request_template", "{\n \"model\": \"{{MODEL}}\",\n \"messages\": {{MESSAGES}}\n}") ?: "{\n \"model\": \"{{MODEL}}\",\n \"messages\": {{MESSAGES}}\n}"
+
+ fun setLlmCustomRequestTemplate(template: String) {
+ sharedPreferences.edit().putString("llm_custom_request_template", template).apply()
+ }
+
+ fun getLlmCustomMessageFormat(): String = sharedPreferences.getString("llm_custom_message_format", "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}") ?: "{\"role\": \"{{ROLE}}\", \"content\": \"{{CONTENT}}\"}"
+
+ fun setLlmCustomMessageFormat(format: String) {
+ sharedPreferences.edit().putString("llm_custom_message_format", format).apply()
+ }
+
+ fun getLlmCustomSystemRole(): String = sharedPreferences.getString("llm_custom_system_role", "system") ?: "system"
+
+ fun setLlmCustomSystemRole(role: String) {
+ sharedPreferences.edit().putString("llm_custom_system_role", role).apply()
+ }
+
+ fun getLlmCustomUserRole(): String = sharedPreferences.getString("llm_custom_user_role", "user") ?: "user"
+
+ fun setLlmCustomUserRole(role: String) {
+ sharedPreferences.edit().putString("llm_custom_user_role", role).apply()
+ }
+
+ fun getLlmCustomAssistantRole(): String = sharedPreferences.getString("llm_custom_assistant_role", "assistant") ?: "assistant"
+
+ fun setLlmCustomAssistantRole(role: String) {
+ sharedPreferences.edit().putString("llm_custom_assistant_role", role).apply()
+ }
+
+ fun getIsImageSupported(): Boolean = sharedPreferences.getBoolean("is_image_supported", false)
+ fun setIsImageSupported(supported: Boolean) {
+ sharedPreferences.edit().putBoolean("is_image_supported", supported).apply()
+ }
+
+ fun getIsAudioSupported(): Boolean = sharedPreferences.getBoolean("is_audio_supported", false)
+ fun setIsAudioSupported(supported: Boolean) {
+ sharedPreferences.edit().putBoolean("is_audio_supported", supported).apply()
+ }
+
+ fun getIsVideoSupported(): Boolean = sharedPreferences.getBoolean("is_video_supported", false)
+ fun setIsVideoSupported(supported: Boolean) {
+ sharedPreferences.edit().putBoolean("is_video_supported", supported).apply()
+ }
+
+ fun getIsDocumentSupported(): Boolean = sharedPreferences.getBoolean("is_document_supported", false)
+ fun setIsDocumentSupported(supported: Boolean) {
+ sharedPreferences.edit().putBoolean("is_document_supported", supported).apply()
+ }
+
+ fun getIsModelVerified(): Boolean = sharedPreferences.getBoolean("is_model_verified", false)
+ fun setIsModelVerified(verified: Boolean) {
+ sharedPreferences.edit().putBoolean("is_model_verified", verified).apply()
+ }
+
+ fun getSttMode(): com.app.assistant.speech.SttMode {
+ val modeStr = sharedPreferences.getString("stt_mode", null)
+ if (modeStr != null) {
+ return try {
+ com.app.assistant.speech.SttMode.valueOf(modeStr)
+ } catch (e: Exception) {
+ com.app.assistant.speech.SttMode.NATIVE
+ }
+ }
+ return if (getUseLocalWhisper()) {
+ com.app.assistant.speech.SttMode.PARAKEET
+ } else {
+ com.app.assistant.speech.SttMode.NATIVE
+ }
+ }
+
+ fun setSttMode(mode: com.app.assistant.speech.SttMode) {
+ sharedPreferences.edit().putString("stt_mode", mode.name).apply()
+ setUseLocalWhisper(mode == com.app.assistant.speech.SttMode.PARAKEET)
+ }
+
+ fun getUseLocalWhisper(): Boolean = sharedPreferences.getBoolean("use_local_whisper", false)
+ fun setUseLocalWhisper(enabled: Boolean) {
+ sharedPreferences.edit().putBoolean("use_local_whisper", enabled).apply()
+ }
+
+ fun getTtsMode(): com.app.assistant.tts.TtsMode {
+ val modeStr = sharedPreferences.getString("tts_mode", com.app.assistant.tts.TtsMode.NATIVE.name)
+ return try {
+ com.app.assistant.tts.TtsMode.valueOf(modeStr ?: com.app.assistant.tts.TtsMode.NATIVE.name)
+ } catch (e: Exception) {
+ com.app.assistant.tts.TtsMode.NATIVE
+ }
+ }
+
+ fun setTtsMode(mode: com.app.assistant.tts.TtsMode) {
+ sharedPreferences.edit().putString("tts_mode", mode.name).apply()
+ }
+}
+
diff --git a/app/src/main/java/com/app/assistant/repository/WeatherRepository.kt b/app/src/main/java/com/app/assistant/repository/WeatherRepository.kt
new file mode 100644
index 0000000..70eacc2
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/repository/WeatherRepository.kt
@@ -0,0 +1,196 @@
+package com.app.assistant.repository
+
+import android.content.Context
+import android.location.Geocoder
+import android.util.Log
+import kotlinx.coroutines.CompletableDeferred
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import kotlinx.serialization.Serializable
+import kotlinx.serialization.json.Json
+import okhttp3.Call
+import okhttp3.Callback
+import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
+import okhttp3.OkHttpClient
+import okhttp3.Request
+import okhttp3.Response
+import java.io.IOException
+import java.util.Locale
+
+@Serializable
+data class GeocodingResult(
+ val name: String,
+ val latitude: Double,
+ val longitude: Double
+)
+
+@Serializable
+data class GeocodingResponse(
+ val results: List? = null
+)
+
+class WeatherRepository(
+ private val context: Context,
+ private val client: OkHttpClient
+) {
+ private val json = Json { ignoreUnknownKeys = true }
+
+ fun extractPlaceName(inputText: String): String {
+ // Pattern to match place names after "in" or "for", excluding trailing words like "today", "tomorrow", etc.
+ val pattern =
+ Regex(
+ "\\b(?:in|for)\\s+([A-Za-z]+(?:\\s+[A-Za-z]+)*)" +
+ "(?=\\b(?:\\s+(today|now|tomorrow|report|forecast|currently|evening|morning|afternoon)\\b|\\s*$))",
+ )
+
+ // Search for a match in the input text
+ val match = pattern.find(inputText)
+
+ // Check if the captured group does not match excluded words (e.g., "tomorrow") and return it
+ val placeName =
+ match
+ ?.groups
+ ?.get(1)
+ ?.value
+ ?.trim()
+ return if (placeName != null &&
+ !placeName.lowercase().matches(Regex("today|now|tomorrow|report|forecast|currently|evening|morning|afternoon"))
+ ) {
+ placeName
+ } else {
+ ""
+ }
+ }
+
+ suspend fun getCoordinatesFromCity(cityName: String): Pair? {
+ val deferredResult = CompletableDeferred?>()
+
+ val urlBuilder =
+ "https://geocoding-api.open-meteo.com/v1/search"
+ .toHttpUrlOrNull()
+ ?.newBuilder()
+ ?.addQueryParameter("name", cityName)
+ ?.build()
+
+ val request =
+ Request
+ .Builder()
+ .url(urlBuilder.toString())
+ .build()
+
+ client.newCall(request).enqueue(
+ object : Callback {
+ override fun onFailure(
+ call: Call,
+ e: IOException,
+ ) {
+ Log.e("WeatherRepository", "Request failed: $e")
+ deferredResult.completeExceptionally(e)
+ }
+
+ override fun onResponse(
+ call: Call,
+ response: Response,
+ ) {
+ if (response.isSuccessful) {
+ val body = response.body?.string() ?: ""
+ deferredResult.complete(extractCoordinates(body, cityName))
+ } else {
+ Log.e("WeatherRepository", "Request unsuccessful: ${response.code}")
+ deferredResult.cancel()
+ }
+ }
+ },
+ )
+
+ return deferredResult.await()
+ }
+
+ private fun extractCoordinates(
+ body: String,
+ cityName: String,
+ ): Pair? {
+ if (body.isEmpty()) return null
+ try {
+ val response = json.decodeFromString(body)
+ val results = response.results ?: return null
+ if (results.isEmpty()) return null
+
+ val matchedResult = results.find { it.name.equals(cityName, ignoreCase = true) } ?: results[0]
+ return Pair(matchedResult.latitude, matchedResult.longitude)
+ } catch (e: Exception) {
+ Log.d("extractCoordinates Exception", e.message ?: "")
+ }
+ return null
+ }
+
+ suspend fun getCityNameFromLocation(
+ latitude: Double,
+ longitude: Double,
+ ): String? = withContext(Dispatchers.IO) {
+ val geocoder = Geocoder(context, Locale.getDefault())
+ return@withContext try {
+ val addresses = geocoder.getFromLocation(latitude, longitude, 1)
+ if (addresses?.isNotEmpty() == true) {
+ addresses[0].locality // This gives the city name
+ } else {
+ null
+ }
+ } catch (e: Exception) {
+ e.printStackTrace()
+ null
+ }
+ }
+
+ suspend fun getWeatherData(
+ latitude: Double,
+ longitude: Double,
+ ): String? {
+ val deferredResult = CompletableDeferred()
+
+ val urlBuilder =
+ "https://api.open-meteo.com/v1/forecast"
+ .toHttpUrlOrNull()
+ ?.newBuilder()
+ ?.addQueryParameter("latitude", latitude.toString())
+ ?.addQueryParameter("longitude", longitude.toString())
+ ?.addQueryParameter("daily", "temperature_2m_max,temperature_2m_min,precipitation_sum,windspeed_10m_max")
+ ?.addQueryParameter("hourly", "temperature_2m,precipitation,windspeed_10m")
+ ?.addQueryParameter("timezone", "auto")
+ ?.addQueryParameter("forecast_days", "10")
+ ?.build()
+
+ val request =
+ Request
+ .Builder()
+ .url(urlBuilder.toString())
+ .build()
+
+ client.newCall(request).enqueue(
+ object : Callback {
+ override fun onFailure(
+ call: Call,
+ e: IOException,
+ ) {
+ Log.e("WeatherRepository", "Weather data request failed: $e")
+ deferredResult.complete(null)
+ }
+
+ override fun onResponse(
+ call: Call,
+ response: Response,
+ ) {
+ if (response.isSuccessful) {
+ val body = response.body?.string() ?: ""
+ deferredResult.complete(body)
+ } else {
+ Log.e("WeatherRepository", "Weather data request unsuccessful: ${response.code}")
+ deferredResult.complete(null)
+ }
+ }
+ },
+ )
+
+ return deferredResult.await()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/speech/AudioHygieneProcessor.kt b/app/src/main/java/com/app/assistant/speech/AudioHygieneProcessor.kt
new file mode 100644
index 0000000..b7537f9
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/speech/AudioHygieneProcessor.kt
@@ -0,0 +1,208 @@
+package com.app.assistant.speech
+
+import android.Manifest
+import android.content.Context
+import android.content.pm.PackageManager
+import android.media.AudioFormat
+import android.media.AudioManager
+import android.media.AudioRecord
+import android.media.MediaRecorder
+import android.media.audiofx.AcousticEchoCanceler
+import android.media.audiofx.AutomaticGainControl
+import android.media.audiofx.NoiseSuppressor
+import android.util.Log
+import androidx.core.app.ActivityCompat
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.delay
+import kotlinx.coroutines.launch
+
+class AudioHygieneProcessor(
+ private val context: Context,
+ private val scope: CoroutineScope,
+ private val isHandsFree: Boolean,
+ private val onSamplesReady: (FloatArray) -> Unit
+) {
+ private val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
+ private var audioRecord: AudioRecord? = null
+ private var recordingJob: Job? = null
+
+ private var echoCanceler: AcousticEchoCanceler? = null
+ private var noiseSuppressor: NoiseSuppressor? = null
+ private var gainControl: AutomaticGainControl? = null
+
+ @Volatile
+ private var isRecording = false
+
+ fun start(): Boolean {
+ if (ActivityCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
+ Log.e("AudioHygieneProcessor", "RECORD_AUDIO permission not granted")
+ return false
+ }
+
+ try {
+ setupAudioRouting()
+
+ val sampleRate = 16000
+ val channelConfig = AudioFormat.CHANNEL_IN_MONO
+ val audioFormat = AudioFormat.ENCODING_PCM_16BIT
+ val bufferSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat)
+
+ val record = AudioRecord(
+ MediaRecorder.AudioSource.VOICE_COMMUNICATION,
+ sampleRate,
+ channelConfig,
+ audioFormat,
+ bufferSize
+ )
+
+ if (record.state != AudioRecord.STATE_INITIALIZED) {
+ Log.e("AudioHygieneProcessor", "Failed to initialize AudioRecord")
+ stopRouting()
+ return false
+ }
+
+ audioRecord = record
+
+ // Explicitly enable system audio effects on the audio session
+ val sessionId = record.audioSessionId
+ enableSystemAudioEffects(sessionId)
+
+ record.startRecording()
+ isRecording = true
+
+ recordingJob = scope.launch(Dispatchers.IO) {
+ val buffer = ShortArray(512)
+ while (isRecording) {
+ val read = record.read(buffer, 0, buffer.size)
+ if (read > 0) {
+ // Convert Short samples to Float samples in range [-1.0, 1.0]
+ val floatSamples = FloatArray(read) { buffer[it] / 32768.0f }
+ onSamplesReady(floatSamples)
+ }
+ //delay(10)
+ }
+ }
+
+ Log.i("AudioHygieneProcessor", "Audio recording started successfully.")
+ return true
+
+ } catch (e: Exception) {
+ Log.e("AudioHygieneProcessor", "Error starting audio recording", e)
+ cleanResources()
+ return false
+ }
+ }
+
+ fun stop() {
+ Log.i("AudioHygieneProcessor", "Stopping audio recording.")
+ isRecording = false
+ recordingJob?.cancel()
+ recordingJob = null
+ cleanResources()
+ }
+
+ private fun setupAudioRouting() {
+ try {
+ if (isHandsFree) {
+ audioManager.mode = AudioManager.MODE_IN_COMMUNICATION
+ audioManager.isSpeakerphoneOn = true
+ Log.d("AudioHygieneProcessor", "Audio routing set to Speakerphone (Hands-Free)")
+ } else {
+ audioManager.mode = AudioManager.MODE_IN_CALL
+ audioManager.isBluetoothScoOn = true
+ audioManager.startBluetoothSco()
+ Log.d("AudioHygieneProcessor", "Audio routing set to Bluetooth SCO")
+ }
+ } catch (e: Exception) {
+ Log.e("AudioHygieneProcessor", "Error setting up audio routing", e)
+ }
+ }
+
+ private fun stopRouting() {
+ try {
+ audioManager.stopBluetoothSco()
+ audioManager.isBluetoothScoOn = false
+ audioManager.isSpeakerphoneOn = false
+ if (audioManager.mode == AudioManager.MODE_IN_CALL || audioManager.mode == AudioManager.MODE_IN_COMMUNICATION) {
+ audioManager.mode = AudioManager.MODE_NORMAL
+ }
+ Log.d("AudioHygieneProcessor", "Audio routing reset to normal")
+ } catch (e: Exception) {
+ Log.e("AudioHygieneProcessor", "Error stopping audio routing", e)
+ }
+ }
+
+ private fun enableSystemAudioEffects(audioSessionId: Int) {
+ // Acoustic Echo Cancellation
+ if (AcousticEchoCanceler.isAvailable()) {
+ try {
+ echoCanceler = AcousticEchoCanceler.create(audioSessionId)?.apply {
+ enabled = true
+ Log.d("AudioHygieneProcessor", "System AcousticEchoCanceler enabled on session $audioSessionId")
+ }
+ } catch (e: Exception) {
+ Log.e("AudioHygieneProcessor", "Failed to create AcousticEchoCanceler", e)
+ }
+ } else {
+ Log.w("AudioHygieneProcessor", "System AcousticEchoCanceler is not available on this device")
+ }
+
+ // Noise Suppression
+ if (NoiseSuppressor.isAvailable()) {
+ try {
+ noiseSuppressor = NoiseSuppressor.create(audioSessionId)?.apply {
+ enabled = true
+ Log.d("AudioHygieneProcessor", "System NoiseSuppressor enabled on session $audioSessionId")
+ }
+ } catch (e: Exception) {
+ Log.e("AudioHygieneProcessor", "Failed to create NoiseSuppressor", e)
+ }
+ } else {
+ Log.w("AudioHygieneProcessor", "System NoiseSuppressor is not available on this device")
+ }
+
+ // Automatic Gain Control
+ if (AutomaticGainControl.isAvailable()) {
+ try {
+ gainControl = AutomaticGainControl.create(audioSessionId)?.apply {
+ enabled = true
+ Log.d("AudioHygieneProcessor", "System AutomaticGainControl enabled on session $audioSessionId")
+ }
+ } catch (e: Exception) {
+ Log.e("AudioHygieneProcessor", "Failed to create AutomaticGainControl", e)
+ }
+ } else {
+ Log.w("AudioHygieneProcessor", "System AutomaticGainControl is not available on this device")
+ }
+ }
+
+ private fun cleanResources() {
+ try {
+ audioRecord?.apply {
+ if (state == AudioRecord.STATE_INITIALIZED) {
+ try {
+ stop()
+ } catch (e: Exception) {
+ Log.e("AudioHygieneProcessor", "Error stopping AudioRecord", e)
+ }
+ }
+ release()
+ }
+ } catch (e: Exception) {
+ Log.e("AudioHygieneProcessor", "Error releasing AudioRecord", e)
+ } finally {
+ audioRecord = null
+ }
+
+ echoCanceler?.release()
+ echoCanceler = null
+ noiseSuppressor?.release()
+ noiseSuppressor = null
+ gainControl?.release()
+ gainControl = null
+
+ stopRouting()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/speech/SpeechModelManager.kt b/app/src/main/java/com/app/assistant/speech/SpeechModelManager.kt
new file mode 100644
index 0000000..20e9193
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/speech/SpeechModelManager.kt
@@ -0,0 +1,197 @@
+package com.app.assistant.speech
+
+import android.content.Context
+import android.util.Log
+import okhttp3.OkHttpClient
+import okhttp3.Request
+import java.io.File
+import java.io.FileOutputStream
+import java.io.IOException
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import java.util.concurrent.atomic.AtomicBoolean
+
+class SpeechModelManager(private val context: Context) {
+
+ private val modelDir = File(context.filesDir, "sherpa-onnx")
+
+ data class ModelFileDesc(
+ val url: String,
+ val localName: String,
+ val sizeBytes: Long
+ )
+
+ private val modelFiles = listOf(
+ ModelFileDesc(
+ url = "https://huggingface.co/csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8/resolve/main/encoder.int8.onnx",
+ localName = "parakeet-encoder.onnx",
+ sizeBytes = 652184296L
+ ),
+ ModelFileDesc(
+ url = "https://huggingface.co/csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8/resolve/main/decoder.int8.onnx",
+ localName = "parakeet-decoder.onnx",
+ sizeBytes = 7257753L
+ ),
+ ModelFileDesc(
+ url = "https://huggingface.co/csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8/resolve/main/joiner.int8.onnx",
+ localName = "parakeet-joiner.onnx",
+ sizeBytes = 1739080L
+ ),
+ ModelFileDesc(
+ url = "https://huggingface.co/csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8/resolve/main/tokens.txt",
+ localName = "parakeet-tokens.txt",
+ sizeBytes = 9384L
+ ),
+ ModelFileDesc(
+ url = "https://huggingface.co/csukuangfj/vad/resolve/main/silero_vad.onnx",
+ localName = "silero_vad.onnx",
+ sizeBytes = 1807522L
+ )
+ )
+
+ val totalSizeBytes: Long = modelFiles.sumOf { it.sizeBytes }
+
+ private var currentCall: okhttp3.Call? = null
+ private val isCancelled = AtomicBoolean(false)
+
+ fun getModelDirectory(): File = modelDir
+
+ fun isModelDownloaded(): Boolean {
+ if (!modelDir.exists()) return false
+ return modelFiles.all { fileDesc ->
+ val file = File(modelDir, fileDesc.localName)
+ file.exists() && file.length() == fileDesc.sizeBytes
+ }
+ }
+
+ fun deleteLocalModel(): Boolean {
+ if (!modelDir.exists()) return true
+ var allDeleted = true
+ modelFiles.forEach { fileDesc ->
+ val file = File(modelDir, fileDesc.localName)
+ if (file.exists()) {
+ val deleted = file.delete()
+ if (!deleted) allDeleted = false
+ }
+ // delete any tmp files too
+ val tmpFile = File(modelDir, "${fileDesc.localName}.tmp")
+ if (tmpFile.exists()) {
+ tmpFile.delete()
+ }
+ }
+ return allDeleted
+ }
+
+ fun getEncoderPath(): String = File(modelDir, "parakeet-encoder.onnx").absolutePath
+ fun getDecoderPath(): String = File(modelDir, "parakeet-decoder.onnx").absolutePath
+ fun getJoinerPath(): String = File(modelDir, "parakeet-joiner.onnx").absolutePath
+ fun getTokensPath(): String = File(modelDir, "parakeet-tokens.txt").absolutePath
+ fun getVadPath(): String = File(modelDir, "silero_vad.onnx").absolutePath
+
+ suspend fun downloadModel(
+ client: OkHttpClient,
+ onProgress: (downloadedBytes: Long, totalBytes: Long) -> Unit,
+ onComplete: (success: Boolean, errorMessage: String?) -> Unit
+ ) {
+ isCancelled.set(false)
+ withContext(Dispatchers.IO) {
+ try {
+ if (!modelDir.exists()) {
+ modelDir.mkdirs()
+ }
+
+ var cumulativeDownloaded = 0L
+
+ for (fileDesc in modelFiles) {
+ val targetFile = File(modelDir, fileDesc.localName)
+
+ // If file is already fully and correctly downloaded, we skip it
+ if (targetFile.exists() && targetFile.length() == fileDesc.sizeBytes) {
+ cumulativeDownloaded += fileDesc.sizeBytes
+ onProgress(cumulativeDownloaded, totalSizeBytes)
+ continue
+ }
+
+ // Otherwise, download it
+ val tmpFile = File(modelDir, "${fileDesc.localName}.tmp")
+ if (tmpFile.exists()) {
+ tmpFile.delete()
+ }
+
+ val request = Request.Builder()
+ .url(fileDesc.url)
+ .build()
+
+ val call = client.newCall(request)
+ currentCall = call
+
+ if (isCancelled.get()) {
+ throw IOException("Download cancelled by user")
+ }
+
+ val response = call.execute()
+ if (!response.isSuccessful) {
+ throw IOException("Failed to download ${fileDesc.localName}: HTTP ${response.code}")
+ }
+
+ val responseBody = response.body ?: throw IOException("Empty response body for ${fileDesc.localName}")
+
+ responseBody.byteStream().use { inputStream ->
+ FileOutputStream(tmpFile).use { outputStream ->
+ val buffer = ByteArray(1024 * 64) // 64kb buffer
+ var bytesRead: Int
+ var fileDownloaded = 0L
+
+ while (inputStream.read(buffer).also { bytesRead = it } != -1) {
+ if (isCancelled.get()) {
+ throw IOException("Download cancelled by user")
+ }
+ outputStream.write(buffer, 0, bytesRead)
+ fileDownloaded += bytesRead
+ onProgress(cumulativeDownloaded + fileDownloaded, totalSizeBytes)
+ }
+ outputStream.flush()
+ }
+ }
+ response.close()
+
+ if (isCancelled.get()) {
+ throw IOException("Download cancelled by user")
+ }
+
+ // Rename tmp to actual
+ if (tmpFile.renameTo(targetFile)) {
+ cumulativeDownloaded += fileDesc.sizeBytes
+ onProgress(cumulativeDownloaded, totalSizeBytes)
+ } else {
+ throw IOException("Failed to rename temporary file for ${fileDesc.localName}")
+ }
+ }
+
+ currentCall = null
+ onComplete(true, null)
+
+ } catch (e: java.lang.Exception) {
+ currentCall = null
+ Log.e("SpeechModelManager", "Error downloading speech models", e)
+
+ // Clean up any remaining .tmp files
+ modelFiles.forEach { fileDesc ->
+ val tmpFile = File(modelDir, "${fileDesc.localName}.tmp")
+ if (tmpFile.exists()) {
+ tmpFile.delete()
+ }
+ }
+
+ val msg = if (isCancelled.get()) "Download cancelled." else e.message ?: "Unknown error"
+ onComplete(false, msg)
+ }
+ }
+ }
+
+ fun cancelDownload() {
+ isCancelled.set(true)
+ currentCall?.cancel()
+ currentCall = null
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/speech/SpeechRecognizerManager.kt b/app/src/main/java/com/app/assistant/speech/SpeechRecognizerManager.kt
new file mode 100644
index 0000000..ac23aab
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/speech/SpeechRecognizerManager.kt
@@ -0,0 +1,689 @@
+package com.app.assistant.speech
+
+import android.Manifest
+import android.content.ComponentCallbacks2
+import android.content.Context
+import android.content.Intent
+import android.content.pm.PackageManager
+import android.content.res.Configuration
+import android.media.AudioFormat
+import android.media.AudioManager
+import android.os.Bundle
+import android.speech.RecognitionListener
+import android.speech.RecognizerIntent
+import android.speech.SpeechRecognizer
+import android.util.Log
+import androidx.core.app.ActivityCompat
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.delay
+import kotlinx.coroutines.launch
+import kotlinx.coroutines.withContext
+import java.util.Locale
+import okhttp3.MultipartBody
+import okhttp3.OkHttpClient
+import okhttp3.Request
+import okhttp3.RequestBody.Companion.toRequestBody
+import okhttp3.MediaType.Companion.toMediaType
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.jsonObject
+import kotlinx.serialization.json.jsonPrimitive
+import java.io.ByteArrayOutputStream
+import com.app.assistant.config.SpeechConfig
+
+class SpeechRecognizerManager(
+ private val context: Context,
+ private val scope: CoroutineScope,
+ private val okHttpClient: okhttp3.OkHttpClient = com.app.assistant.viewmodel.MainViewModelFactory.okHttpClient,
+) {
+ private val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
+ private val settingsRepository = com.app.assistant.repository.SettingsRepository(context)
+ private var speechRecognizer: android.speech.SpeechRecognizer? = null
+ private var isListening = false
+
+ var shouldIgnoreUiHidden: () -> Boolean = { false }
+
+ fun isListeningActive(): Boolean = isListening
+
+ private var vad: com.k2fsa.sherpa.onnx.Vad? = null
+ private var offlineRecognizer: com.k2fsa.sherpa.onnx.OfflineRecognizer? = null
+
+ private val modelManager = SpeechModelManager(context)
+ private var isInHybridTransition = false
+ private var hybridPrefixText = ""
+
+ private var activeListener: SpeechListener? = null
+ private var isHandsFreeMode = false
+ @Volatile
+ private var isMicMuted = false
+
+ @Volatile
+ var speechStartTimestamp: Long = 0L
+
+ private var audioHygieneProcessor: AudioHygieneProcessor? = null
+ private var vadIntelligenceProcessor: VadIntelligenceProcessor? = null
+ private var voiceStateMachine: VoiceStateMachine? = null
+
+ @Volatile
+ var isTtsSpeaking = false
+ set(value) {
+ field = value
+ voiceStateMachine?.onTtsStateChanged(value)
+ }
+
+ private val componentCallbacks = object : ComponentCallbacks2 {
+ override fun onTrimMemory(level: Int) {
+ if (level == ComponentCallbacks2.TRIM_MEMORY_UI_HIDDEN) {
+ if (!shouldIgnoreUiHidden()) {
+ stop()
+ }
+ }
+ }
+ override fun onConfigurationChanged(newConfig: Configuration) {}
+ override fun onLowMemory() {}
+ }
+
+ init {
+ context.registerComponentCallbacks(componentCallbacks)
+ }
+
+ interface SpeechListener {
+ fun onReadyForSpeech()
+ fun onBeginningOfSpeech()
+ fun onEndOfSpeech()
+ fun onError(errorCode: Int)
+ fun onResults(recognizedText: String)
+ fun onPartialResults(recognizedText: String)
+ fun onAudioLevelChanged(level: Float) {}
+ }
+
+ @Synchronized
+ private fun initVad() {
+ if (vad != null) return
+ try {
+ if (!modelManager.isModelDownloaded()) {
+ Log.w("SpeechRecognizerManager", "Cannot init VAD: Model files not downloaded.")
+ return
+ }
+
+ val sileroConfig = com.k2fsa.sherpa.onnx.SileroVadModelConfig(
+ model = modelManager.getVadPath(),
+ threshold = 0.5f,
+ minSilenceDuration = 0.5f,
+ minSpeechDuration = 0.1f,
+ windowSize = 512,
+ maxSpeechDuration = 10.0f
+ )
+ val vadConfig = com.k2fsa.sherpa.onnx.VadModelConfig(
+ sileroVadModelConfig = sileroConfig,
+ sampleRate = 16000,
+ numThreads = 4,
+ provider = "cpu"
+ )
+ vad = com.k2fsa.sherpa.onnx.Vad(
+ assetManager = null,
+ config = vadConfig
+ )
+ Log.i("SpeechRecognizerManager", "[Silero VAD] Initialized successfully")
+ } catch (e: Exception) {
+ Log.e("SpeechRecognizerManager", "Failed to initialize VAD from disk", e)
+ }
+ }
+
+ @Synchronized
+ private fun initOfflineRecognizer() {
+ if (offlineRecognizer != null) return
+ try {
+ if (!modelManager.isModelDownloaded()) {
+ Log.w("SpeechRecognizerManager", "Cannot init Parakeet: Model files not downloaded.")
+ return
+ }
+
+ val transducerConfig = com.k2fsa.sherpa.onnx.OfflineTransducerModelConfig(
+ encoder = modelManager.getEncoderPath(),
+ decoder = modelManager.getDecoderPath(),
+ joiner = modelManager.getJoinerPath()
+ )
+ val modelConfig = com.k2fsa.sherpa.onnx.OfflineModelConfig(
+ transducer = transducerConfig,
+ tokens = modelManager.getTokensPath(),
+ modelType = "nemo_transducer",
+ numThreads = 2,
+ provider = "cpu"
+ )
+ val recognizerConfig = com.k2fsa.sherpa.onnx.OfflineRecognizerConfig(
+ modelConfig = modelConfig,
+ featConfig = com.k2fsa.sherpa.onnx.FeatureConfig(
+ sampleRate = 16000,
+ featureDim = 128
+ ),
+ decodingMethod = "greedy_search"
+ )
+ offlineRecognizer = com.k2fsa.sherpa.onnx.OfflineRecognizer(
+ assetManager = null,
+ config = recognizerConfig
+ )
+ Log.i("SpeechRecognizerManager", "[Parakeet Recognizer] Initialized successfully")
+ } catch (e: Exception) {
+ Log.e("SpeechRecognizerManager", "Failed to initialize Parakeet from disk", e)
+ }
+ }
+
+ fun preLoadModelAsync() {
+ val mode = settingsRepository.getSttMode()
+ if (mode == SttMode.PARAKEET && modelManager.isModelDownloaded()) {
+ scope.launch(Dispatchers.IO) {
+ initVad()
+ initOfflineRecognizer()
+ }
+ }
+ }
+
+ fun startListening(isHandsFree: Boolean = false, listener: SpeechListener) {
+ val mode = settingsRepository.getSttMode()
+ Log.i("SpeechRecognizerManager", "[Listening] Start Listening requested. Hands-free: $isHandsFree, STT Mode: $mode")
+
+ isInHybridTransition = false
+ hybridPrefixText = ""
+ activeListener = listener
+ isHandsFreeMode = isHandsFree
+
+ if (isHandsFree) {
+ if (modelManager.isModelDownloaded()) {
+ startThreeLayerPipeline(isHandsFree = true, listener)
+ } else {
+ Log.w("SpeechRecognizerManager", "Hands-free requested but model not downloaded. Falling back to Native STT.")
+ startNativeListening(listener)
+ }
+ return
+ }
+
+ when (mode) {
+ SttMode.NATIVE -> {
+ startNativeListening(listener)
+ }
+ SttMode.PARAKEET -> {
+ if (modelManager.isModelDownloaded()) {
+ startThreeLayerPipeline(isHandsFree = false, listener)
+ } else {
+ Log.w("SpeechRecognizerManager", "Parakeet selected but not downloaded. Falling back to Native STT.")
+ startNativeListening(listener)
+ }
+ }
+ SttMode.HYBRID -> {
+ if (modelManager.isModelDownloaded()) {
+ if (offlineRecognizer != null && vad != null) {
+ startThreeLayerPipeline(isHandsFree = false, listener)
+ } else {
+ startHybridTransitionListening(listener)
+ }
+ } else {
+ Log.w("SpeechRecognizerManager", "Hybrid selected but model not downloaded. Using Native STT.")
+ startNativeListening(listener)
+ }
+ }
+ SttMode.API -> {
+ if (modelManager.isModelDownloaded()) {
+ startThreeLayerPipeline(isHandsFree = isHandsFree, listener)
+ } else {
+ Log.w("SpeechRecognizerManager", "STT API selected but VAD model not downloaded. Falling back to Native STT.")
+ startNativeListening(listener)
+ }
+ }
+ }
+ }
+
+ private val stateMachineCallback = object : VoiceStateMachine.StateCallback {
+ override fun onTransitionToListening() {
+ scope.launch(Dispatchers.Main) {
+ isListening = true
+ activeListener?.onBeginningOfSpeech()
+ }
+ }
+
+ override fun onTransitionToProcessing(speechSamples: FloatArray) {
+ scope.launch(Dispatchers.Main) {
+ isListening = false
+ activeListener?.onEndOfSpeech()
+ }
+ scope.launch(Dispatchers.IO) {
+ transcribeAndDeliver(speechSamples)
+ }
+ }
+
+ override fun onTransitionToBotSpeaking() {
+ isListening = false
+ vadIntelligenceProcessor?.clear()
+ }
+
+ override fun onTransitionToIdle() {
+ isListening = false
+ stopPipeline()
+ }
+
+ override fun stopTtsPlayback() {
+ // This propagates TTS state change
+ isTtsSpeaking = false
+ }
+
+ override fun onMicReady(ready: Boolean) {
+ scope.launch(Dispatchers.Main) {
+ if (ready) {
+ activeListener?.onReadyForSpeech()
+ }
+ }
+ }
+ }
+
+ private fun startThreeLayerPipeline(isHandsFree: Boolean, listener: SpeechListener) {
+ scope.launch {
+ try {
+ cleanupSpeechRecognizer()
+ withContext(Dispatchers.IO) {
+ initVad()
+ val mode = settingsRepository.getSttMode()
+ if (mode == SttMode.PARAKEET || mode == SttMode.HYBRID || mode == SttMode.NATIVE) {
+ initOfflineRecognizer()
+ }
+
+ val activeVad = vad
+ if (activeVad == null) {
+ Log.e("SpeechRecognizerManager", "VAD is null. Cannot listen.")
+ withContext(Dispatchers.Main) {
+ listener.onError(-1)
+ }
+ return@withContext
+ }
+
+ // Stop any existing pipeline before starting a new one
+ stopPipeline()
+
+ // Initialize the Orchestrator (Layer 3)
+ val stateMachine = VoiceStateMachine(isHandsFree, stateMachineCallback)
+ voiceStateMachine = stateMachine
+
+ // Initialize the Intelligence Layer (Layer 2)
+ val vadProcessor = VadIntelligenceProcessor(activeVad, object : VadIntelligenceProcessor.VadListener {
+ override fun onSpeechStart() {
+ speechStartTimestamp = System.currentTimeMillis()
+ stateMachine.onSpeechStartDetected()
+ }
+
+ override fun onSpeechEnd(samples: FloatArray) {
+ stateMachine.onSpeechEndDetected(samples)
+ }
+ })
+ vadIntelligenceProcessor = vadProcessor
+
+ // Initialize the Audio Hygiene Layer (Layer 1)
+ val audioProcessor = AudioHygieneProcessor(context, scope, isHandsFree) { floatSamples ->
+ val rms = calculateRMS(floatSamples)
+ val normalized = (rms / 0.15f).coerceIn(0f, 1f)
+ scope.launch(Dispatchers.Main) {
+ listener.onAudioLevelChanged(normalized)
+ }
+ // Pass samples directly up to Layer 2
+ vadProcessor.acceptSamples(floatSamples)
+ }
+ audioHygieneProcessor = audioProcessor
+
+ vadProcessor.clear()
+ if (isMicMuted) {
+ Log.i("SpeechRecognizerManager", "Pipeline initialized but mic is muted. Not starting AudioRecord.")
+ stateMachine.onUserStartedListening()
+ } else {
+ if (audioProcessor.start()) {
+ stateMachine.onUserStartedListening()
+ } else {
+ withContext(Dispatchers.Main) {
+ listener.onError(-1)
+ }
+ }
+ }
+ }
+ } catch (e: Exception) {
+ Log.e("SpeechRecognizerManager", "Error starting 3-layer pipeline", e)
+ withContext(Dispatchers.Main) {
+ listener.onError(-1)
+ }
+ }
+ }
+ }
+
+ private suspend fun transcribeAndDeliver(samples: FloatArray) {
+ val currentListener = activeListener ?: return
+ val mode = settingsRepository.getSttMode()
+ var text = ""
+
+ try {
+ if (mode == SttMode.PARAKEET || mode == SttMode.HYBRID || mode == SttMode.NATIVE) {
+ val recognizer = offlineRecognizer
+ if (recognizer != null) {
+ val stream = recognizer.createStream()
+ stream.acceptWaveform(samples, 16000)
+ recognizer.decode(stream)
+ text = recognizer.getResult(stream).text.trim()
+ stream.release()
+ } else {
+ Log.e("SpeechRecognizerManager", "Offline recognizer not initialized")
+ }
+ } else if (mode == SttMode.API) {
+ // Convert float samples back to short samples for WAV encoding
+ val shortSamples = ShortArray(samples.size) {
+ (samples[it] * 32767.0f).toInt().coerceIn(-32768, 32767).toShort()
+ }
+ val wavBytes = getWavBytes(shortSamples, 16000)
+ text = transcribeWithGroq(wavBytes)
+ }
+
+ withContext(Dispatchers.Main) {
+ if (text.isNotEmpty()) {
+ currentListener.onResults(text)
+ } else {
+ currentListener.onError(SpeechRecognizer.ERROR_NO_MATCH)
+ }
+ }
+ } catch (e: Exception) {
+ Log.e("SpeechRecognizerManager", "Transcription failed", e)
+ withContext(Dispatchers.Main) {
+ currentListener.onError(-1)
+ }
+ } finally {
+ if (!isHandsFreeMode) {
+ stop()
+ }
+ }
+ }
+
+ private fun startHybridTransitionListening(listener: SpeechListener) {
+ isInHybridTransition = false
+ hybridPrefixText = ""
+ startNativeListening(listener)
+
+ scope.launch(Dispatchers.IO) {
+ initVad()
+ initOfflineRecognizer()
+ }
+ }
+
+ private fun startNativeListening(listener: SpeechListener) {
+ try {
+ cleanupSpeechRecognizer()
+ if (isMicMuted) {
+ Log.i("SpeechRecognizerManager", "startNativeListening: mic is muted, not starting SpeechRecognizer")
+ return
+ }
+
+ audioManager.mode = AudioManager.MODE_IN_CALL
+ audioManager.isBluetoothScoOn = true
+ audioManager.startBluetoothSco()
+
+ val recognizer = SpeechRecognizer.createSpeechRecognizer(context)
+ speechRecognizer = recognizer
+
+ val speechRecognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH).apply {
+ putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM)
+ putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true)
+ putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault())
+ }
+
+ recognizer.setRecognitionListener(object : RecognitionListener {
+ override fun onReadyForSpeech(bundle: Bundle?) {
+ listener.onReadyForSpeech()
+ }
+
+ override fun onBeginningOfSpeech() {
+ speechStartTimestamp = System.currentTimeMillis()
+ isListening = true
+ listener.onBeginningOfSpeech()
+ }
+
+ override fun onRmsChanged(v: Float) {
+ val normalized = ((v + 2f) / 12f).coerceIn(0f, 1f)
+ listener.onAudioLevelChanged(normalized)
+ }
+ override fun onBufferReceived(bytes: ByteArray?) {}
+
+ override fun onEndOfSpeech() {
+ isListening = false
+ listener.onEndOfSpeech()
+ }
+
+ override fun onError(errorCode: Int) {
+ isListening = false
+ listener.onError(errorCode)
+ stopBluetoothSco()
+ }
+
+ override fun onResults(bundle: Bundle?) {
+ isListening = false
+ val recognizedText = bundle?.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)?.get(0) ?: ""
+ listener.onResults(recognizedText)
+ stopBluetoothSco()
+ }
+
+ override fun onPartialResults(bundle: Bundle) {
+ val recognizedText = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)?.get(0) ?: ""
+ listener.onPartialResults(recognizedText)
+ }
+
+ override fun onEvent(i: Int, bundle: Bundle?) {}
+ })
+
+ recognizer.startListening(speechRecognizerIntent)
+ } catch (e: Exception) {
+ Log.e("SpeechRecognizerManager", "Error starting native speech recognition", e)
+ stopBluetoothSco()
+ listener.onError(-1)
+ }
+ }
+
+ private fun stopBluetoothSco() {
+ try {
+ audioManager.stopBluetoothSco()
+ audioManager.isBluetoothScoOn = false
+ audioManager.isSpeakerphoneOn = false
+ if (audioManager.mode == AudioManager.MODE_IN_CALL || audioManager.mode == AudioManager.MODE_IN_COMMUNICATION) {
+ audioManager.mode = AudioManager.MODE_NORMAL
+ }
+ } catch (e: Exception) {
+ Log.e("SpeechRecognizerManager", "Error stopping Bluetooth SCO", e)
+ }
+ }
+
+ private fun stopPipeline() {
+ audioHygieneProcessor?.stop()
+ audioHygieneProcessor = null
+
+ vadIntelligenceProcessor?.clear()
+ vadIntelligenceProcessor = null
+
+ voiceStateMachine = null
+ }
+
+ @Synchronized
+ fun setMicMuted(muted: Boolean) {
+ if (isMicMuted == muted) return
+ isMicMuted = muted
+ Log.i("SpeechRecognizerManager", "setMicMuted changed to $muted")
+
+ if (muted) {
+ stopMicAccess()
+ } else {
+ resumeMicAccess()
+ }
+ }
+
+ private fun stopMicAccess() {
+ Log.i("SpeechRecognizerManager", "stopMicAccess: making microphone unavailable")
+ audioHygieneProcessor?.stop()
+
+ speechRecognizer?.let {
+ it.stopListening()
+ cleanupSpeechRecognizer()
+ stopBluetoothSco()
+ }
+ }
+
+ private fun resumeMicAccess() {
+ Log.i("SpeechRecognizerManager", "resumeMicAccess: resuming microphone access")
+ val listener = activeListener ?: return
+
+ if (isHandsFreeMode) {
+ val audioProcessor = audioHygieneProcessor
+ val stateMachine = voiceStateMachine
+ if (audioProcessor != null && stateMachine != null) {
+ scope.launch(Dispatchers.IO) {
+ if (audioProcessor.start()) {
+ stateMachine.onUserStartedListening()
+ } else {
+ scope.launch(Dispatchers.Main) {
+ listener.onError(-1)
+ }
+ }
+ }
+ } else {
+ if (modelManager.isModelDownloaded()) {
+ startThreeLayerPipeline(isHandsFree = true, listener)
+ } else {
+ startNativeListening(listener)
+ }
+ }
+ }
+ }
+
+ fun stop() {
+ Log.i("SpeechRecognizerManager", "[Listening] Stop Listening requested manually")
+ voiceStateMachine?.onUserStoppedListening()
+ stopPipeline()
+ cleanupSpeechRecognizer()
+ stopBluetoothSco()
+ }
+
+ private fun cleanupSpeechRecognizer() {
+ speechRecognizer?.let {
+ it.destroy()
+ speechRecognizer = null
+ }
+ }
+
+ fun destroy() {
+ Log.i("SpeechRecognizerManager", "[Listening] destroy() requested")
+ context.unregisterComponentCallbacks(componentCallbacks)
+ stop()
+
+ vad?.let {
+ it.release()
+ vad = null
+ Log.i("SpeechRecognizerManager", "[Silero VAD] Released/Stopped")
+ }
+
+ offlineRecognizer?.let {
+ it.release()
+ offlineRecognizer = null
+ Log.i("SpeechRecognizerManager", "[Parakeet Recognizer] Released/Stopped")
+ }
+ }
+
+ private fun getWavBytes(pcmData: ShortArray, sampleRate: Int): ByteArray {
+ val totalAudioLen = pcmData.size * 2
+ val totalDataLen = totalAudioLen + 36
+ val longSampleRate = sampleRate.toLong()
+ val channels = 1
+ val byteRate = sampleRate * 2
+
+ val header = ByteArray(44)
+ header[0] = 'R'.code.toByte()
+ header[1] = 'I'.code.toByte()
+ header[2] = 'F'.code.toByte()
+ header[3] = 'F'.code.toByte()
+ header[4] = (totalDataLen and 0xff).toByte()
+ header[5] = ((totalDataLen shr 8) and 0xff).toByte()
+ header[6] = ((totalDataLen shr 16) and 0xff).toByte()
+ header[7] = ((totalDataLen shr 24) and 0xff).toByte()
+ header[8] = 'W'.code.toByte()
+ header[9] = 'A'.code.toByte()
+ header[10] = 'V'.code.toByte()
+ header[11] = 'E'.code.toByte()
+ header[12] = 'f'.code.toByte()
+ header[13] = 'm'.code.toByte()
+ header[14] = 't'.code.toByte()
+ header[15] = ' '.code.toByte()
+ header[16] = 16
+ header[17] = 0
+ header[18] = 0
+ header[19] = 0
+ header[20] = 1
+ header[21] = 0
+ header[22] = channels.toByte()
+ header[23] = 0
+ header[24] = (longSampleRate and 0xff).toByte()
+ header[25] = ((longSampleRate shr 8) and 0xff).toByte()
+ header[26] = ((longSampleRate shr 16) and 0xff).toByte()
+ header[27] = ((longSampleRate shr 24) and 0xff).toByte()
+ header[28] = (byteRate and 0xff).toByte()
+ header[29] = ((byteRate shr 8) and 0xff).toByte()
+ header[30] = ((byteRate shr 16) and 0xff).toByte()
+ header[31] = ((byteRate shr 24) and 0xff).toByte()
+ header[32] = 2
+ header[33] = 0
+ header[34] = 16
+ header[35] = 0
+ header[36] = 'd'.code.toByte()
+ header[37] = 'a'.code.toByte()
+ header[38] = 't'.code.toByte()
+ header[39] = 'a'.code.toByte()
+ header[40] = (totalAudioLen and 0xff).toByte()
+ header[41] = ((totalAudioLen shr 8) and 0xff).toByte()
+ header[42] = ((totalAudioLen shr 16) and 0xff).toByte()
+ header[43] = ((totalAudioLen shr 24) and 0xff).toByte()
+
+ val wavBytes = ByteArray(44 + totalAudioLen)
+ System.arraycopy(header, 0, wavBytes, 0, 44)
+
+ var offset = 44
+ for (sample in pcmData) {
+ wavBytes[offset++] = (sample.toInt() and 0xff).toByte()
+ wavBytes[offset++] = ((sample.toInt() shr 8) and 0xff).toByte()
+ }
+ return wavBytes
+ }
+
+ private suspend fun transcribeWithGroq(wavBytes: ByteArray): String = withContext(Dispatchers.IO) {
+ val requestBody = okhttp3.MultipartBody.Builder()
+ .setType(okhttp3.MultipartBody.FORM)
+ .addFormDataPart("model", com.app.assistant.config.SpeechConfig.GroqWhisper.MODEL)
+ .addFormDataPart(
+ "file",
+ "speech.wav",
+ wavBytes.toRequestBody("audio/wav".toMediaType())
+ )
+ .build()
+
+ val apiKey = com.app.assistant.config.SpeechConfig.GroqWhisper.getApiKey(settingsRepository)
+ val request = okhttp3.Request.Builder()
+ .url(com.app.assistant.config.SpeechConfig.GroqWhisper.BASE_URL)
+ .addHeader("Authorization", "Bearer $apiKey")
+ .post(requestBody)
+ .build()
+
+ okHttpClient.newCall(request).execute().use { response ->
+ if (!response.isSuccessful) {
+ throw Exception("Unsuccessful response from Groq: ${response.code} - ${response.message}")
+ }
+ val bodyStr = response.body?.string() ?: ""
+ val jsonElement = kotlinx.serialization.json.Json.parseToJsonElement(bodyStr)
+ jsonElement.jsonObject["text"]?.jsonPrimitive?.content ?: ""
+ }
+ }
+
+ private fun calculateRMS(samples: FloatArray): Float {
+ if (samples.isEmpty()) return 0f
+ var sum = 0.0f
+ for (sample in samples) {
+ sum += sample * sample
+ }
+ return kotlin.math.sqrt(sum / samples.size)
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/speech/SttMode.kt b/app/src/main/java/com/app/assistant/speech/SttMode.kt
new file mode 100644
index 0000000..703f6eb
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/speech/SttMode.kt
@@ -0,0 +1,8 @@
+package com.app.assistant.speech
+
+enum class SttMode {
+ NATIVE,
+ PARAKEET,
+ HYBRID,
+ API
+}
diff --git a/app/src/main/java/com/app/assistant/speech/VadIntelligenceProcessor.kt b/app/src/main/java/com/app/assistant/speech/VadIntelligenceProcessor.kt
new file mode 100644
index 0000000..f0c112d
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/speech/VadIntelligenceProcessor.kt
@@ -0,0 +1,98 @@
+package com.app.assistant.speech
+
+import android.util.Log
+import com.k2fsa.sherpa.onnx.Vad
+import kotlin.math.max
+import kotlin.math.sqrt
+
+class VadIntelligenceProcessor(
+ private val vad: Vad,
+ private val listener: VadListener
+) {
+ interface VadListener {
+ fun onSpeechStart()
+ fun onSpeechEnd(samples: FloatArray)
+ }
+
+ private var isSpeechActive = false
+
+ private var ambientNoiseFloor = 0.002f
+
+ companion object {
+ private const val SAMPLE_RATE = 16000
+ private const val MIN_SPEECH_DURATION_SEC = 0.4f
+ private const val MIN_SAMPLES_REQUIRED = (SAMPLE_RATE * MIN_SPEECH_DURATION_SEC).toInt()
+
+ private const val ABSOLUTE_MIN_THRESHOLD = 0.0015f
+
+ private const val SNR_MULTIPLIER = 1.8f
+
+ private const val EMA_ALPHA = 0.05f
+ }
+
+ fun acceptSamples(samples: FloatArray) {
+ try {
+ vad.acceptWaveform(samples)
+
+ val currentSpeechDetected = vad.isSpeechDetected()
+
+ if (!currentSpeechDetected && samples.isNotEmpty()) {
+ val chunkRms = calculateRMS(samples)
+ ambientNoiseFloor = (ambientNoiseFloor * (1f - EMA_ALPHA)) + (chunkRms * EMA_ALPHA)
+ }
+
+ if (currentSpeechDetected && !isSpeechActive) {
+ isSpeechActive = true
+ Log.d("VadIntelligenceProcessor", "VAD Speech start transition detected!")
+ listener.onSpeechStart()
+ } else if (!currentSpeechDetected && isSpeechActive) {
+ isSpeechActive = false
+ Log.d("VadIntelligenceProcessor", "VAD Speech end transition detected!")
+ }
+
+ while (!vad.empty()) {
+ val segment = vad.front()
+ vad.pop()
+
+ val audioSamples = segment.samples
+
+ if (audioSamples.size < MIN_SAMPLES_REQUIRED) {
+ Log.d("VadIntelligenceProcessor", "Rejected segment: Too short (${audioSamples.size} samples)")
+ continue
+ }
+
+ val rms = calculateRMS(audioSamples)
+ val dynamicThreshold = max(ABSOLUTE_MIN_THRESHOLD, ambientNoiseFloor * SNR_MULTIPLIER)
+
+ if (rms < dynamicThreshold) {
+ Log.d("VadIntelligenceProcessor", "Rejected segment: Low energy (RMS: $rms, Dynamic Threshold: $dynamicThreshold)")
+ continue
+ }
+
+ Log.i("VadIntelligenceProcessor", "Valid speech verified. Samples: ${audioSamples.size}, RMS: $rms, Threshold: $dynamicThreshold")
+ listener.onSpeechEnd(audioSamples)
+ }
+ } catch (e: Exception) {
+ Log.e("VadIntelligenceProcessor", "Error processing VAD acceptWaveform", e)
+ }
+ }
+
+ private fun calculateRMS(samples: FloatArray): Float {
+ if (samples.isEmpty()) return 0f
+ var sum = 0.0f
+ for (sample in samples) {
+ sum += sample * sample
+ }
+ return sqrt(sum / samples.size)
+ }
+
+ fun clear() {
+ try {
+ vad.clear()
+ isSpeechActive = false
+ Log.d("VadIntelligenceProcessor", "VAD state cleared.")
+ } catch (e: Exception) {
+ Log.e("VadIntelligenceProcessor", "Error clearing VAD", e)
+ }
+ }
+}
\ No newline at end of file
diff --git a/app/src/main/java/com/app/assistant/speech/VoiceState.kt b/app/src/main/java/com/app/assistant/speech/VoiceState.kt
new file mode 100644
index 0000000..d6bcb5f
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/speech/VoiceState.kt
@@ -0,0 +1,8 @@
+package com.app.assistant.speech
+
+enum class VoiceState {
+ IDLE,
+ LISTENING,
+ PROCESSING,
+ BOT_SPEAKING
+}
diff --git a/app/src/main/java/com/app/assistant/speech/VoiceStateMachine.kt b/app/src/main/java/com/app/assistant/speech/VoiceStateMachine.kt
new file mode 100644
index 0000000..2ebc177
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/speech/VoiceStateMachine.kt
@@ -0,0 +1,125 @@
+package com.app.assistant.speech
+
+class VoiceStateMachine(
+ private val isHandsFree: Boolean,
+ private val callback: StateCallback
+) {
+ interface StateCallback {
+ fun onTransitionToListening()
+ fun onTransitionToProcessing(speechSamples: FloatArray)
+ fun onTransitionToBotSpeaking()
+ fun onTransitionToIdle()
+ fun stopTtsPlayback()
+ fun onMicReady(ready: Boolean)
+ }
+
+ private var state = VoiceState.IDLE
+ private var lastTtsStartTime: Long = 0L
+
+ @Synchronized
+ fun getCurrentState(): VoiceState = state
+
+ @Synchronized
+ fun onUserStartedListening() {
+ if (state == VoiceState.IDLE) {
+ state = VoiceState.LISTENING
+ logD("State transition: IDLE -> LISTENING")
+ callback.onMicReady(true)
+ callback.onTransitionToListening()
+ }
+ }
+
+ @Synchronized
+ fun onUserStoppedListening() {
+ if (state != VoiceState.IDLE) {
+ logD("State transition: $state -> IDLE (stopped manually)")
+ state = VoiceState.IDLE
+ callback.onMicReady(false)
+ callback.onTransitionToIdle()
+ }
+ }
+
+ @Synchronized
+ fun onSpeechStartDetected() {
+ when (state) {
+ VoiceState.BOT_SPEAKING -> {
+ val elapsedSinceTtsStart = System.currentTimeMillis() - lastTtsStartTime
+ if (elapsedSinceTtsStart < 300) {
+ logD("Ignoring speech start detected during echo bleed window ($elapsedSinceTtsStart ms < 300 ms).")
+ return
+ }
+ logI("[Interruption] Speech detected during TTS. Stopping TTS and listening.")
+ callback.stopTtsPlayback()
+ state = VoiceState.LISTENING
+ callback.onMicReady(true)
+ callback.onTransitionToListening()
+ }
+ VoiceState.LISTENING -> {
+ logD("VAD Speech start detected while listening.")
+ callback.onTransitionToListening()
+ }
+ else -> {
+ logD("VAD Speech start detected during state: $state. Ignoring.")
+ }
+ }
+ }
+
+ @Synchronized
+ fun onSpeechEndDetected(samples: FloatArray) {
+ if (state == VoiceState.LISTENING || state == VoiceState.BOT_SPEAKING) {
+ logD("State transition: $state -> PROCESSING")
+ state = VoiceState.PROCESSING
+ callback.onMicReady(false)
+ callback.onTransitionToProcessing(samples)
+ } else {
+ logD("VAD Speech end detected during state: $state. Ignoring.")
+ }
+ }
+
+ @Synchronized
+ fun onTtsStateChanged(isSpeaking: Boolean) {
+ logD("TTS state changed: isSpeaking = $isSpeaking, current state = $state")
+ when {
+ isSpeaking -> {
+ lastTtsStartTime = System.currentTimeMillis()
+ if (state == VoiceState.PROCESSING || state == VoiceState.LISTENING) {
+ logD("State transition: $state -> BOT_SPEAKING")
+ state = VoiceState.BOT_SPEAKING
+ callback.onTransitionToBotSpeaking()
+ }
+ }
+ else -> {
+ // TTS stopped
+ if (state == VoiceState.BOT_SPEAKING) {
+ if (isHandsFree) {
+ logD("State transition: BOT_SPEAKING -> LISTENING (Hands-Free auto-loop)")
+ state = VoiceState.LISTENING
+ callback.onMicReady(true)
+ callback.onTransitionToListening()
+ } else {
+ logD("State transition: BOT_SPEAKING -> IDLE")
+ state = VoiceState.IDLE
+ callback.onMicReady(false)
+ callback.onTransitionToIdle()
+ }
+ }
+ }
+ }
+ }
+
+ private fun logD(msg: String) {
+ try {
+ android.util.Log.d("VoiceStateMachine", msg)
+ } catch (e: Throwable) {
+ println("[VoiceStateMachine] D: $msg")
+ }
+ }
+
+ private fun logI(msg: String) {
+ try {
+ android.util.Log.i("VoiceStateMachine", msg)
+ } catch (e: Throwable) {
+ println("[VoiceStateMachine] I: $msg")
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/translation/TranslatorManager.kt b/app/src/main/java/com/app/assistant/translation/TranslatorManager.kt
index 6e98f65..c25d4a1 100644
--- a/app/src/main/java/com/app/assistant/translation/TranslatorManager.kt
+++ b/app/src/main/java/com/app/assistant/translation/TranslatorManager.kt
@@ -10,51 +10,61 @@ import com.google.mlkit.nl.translate.Translator
import com.google.mlkit.nl.translate.TranslatorOptions
class TranslatorManager {
-
private var toEnglishTranslator: Translator? = null
private var fromEnglishTranslator: Translator? = null
private val modelManager = RemoteModelManager.getInstance()
- fun setupTranslators(selectedLanguageCode: String, onResult: (Boolean) -> Unit) {
- modelManager.getDownloadedModels(TranslateRemoteModel::class.java)
+ fun setupTranslators(
+ selectedLanguageCode: String,
+ onResult: (Boolean) -> Unit,
+ ) {
+ modelManager
+ .getDownloadedModels(TranslateRemoteModel::class.java)
.addOnSuccessListener { downloadedModels ->
val downloadedLanguages = downloadedModels.map { it.language }.toSet()
val isSelectedModelDownloaded = selectedLanguageCode in downloadedLanguages
val isEnglishModelDownloaded = TranslateLanguage.ENGLISH in downloadedLanguages
- // Download selected language model if not already downloaded
if (!isSelectedModelDownloaded) {
downloadLanguageModel(selectedLanguageCode) { success ->
if (!success) {
onResult(false)
- return@downloadLanguageModel
- }else{
- // Download English model if not already downloaded
+ } else {
if (!isEnglishModelDownloaded) {
downloadLanguageModel(TranslateLanguage.ENGLISH) { successEnglish ->
if (!successEnglish) {
onResult(false)
- return@downloadLanguageModel
- }else{
+ } else {
+ initializeTranslators(selectedLanguageCode)
onResult(true)
}
}
} else {
Log.d("Translator", "English language model already exists. Not downloading.")
+ initializeTranslators(selectedLanguageCode)
onResult(true)
}
}
}
} else {
- Log.d("Translator", "Selected language model ($selectedLanguageCode) already exists. Not downloading.")
- onResult(true)
+ if (!isEnglishModelDownloaded) {
+ Log.d("Translator", "Selected model exists, downloading English model.")
+ downloadLanguageModel(TranslateLanguage.ENGLISH) { successEnglish ->
+ if (!successEnglish) {
+ onResult(false)
+ } else {
+ initializeTranslators(selectedLanguageCode)
+ onResult(true)
+ }
+ }
+ } else {
+ Log.d("Translator", "Both models already exist. Not downloading.")
+ initializeTranslators(selectedLanguageCode)
+ onResult(true)
+ }
}
-
- // Set up the translators for the selected language and English
- initializeTranslators(selectedLanguageCode)
- }
- .addOnFailureListener { e ->
+ }.addOnFailureListener { e ->
Log.e("Translator", "Failed to get downloaded models: ${e.message}")
onResult(false)
}
@@ -63,57 +73,69 @@ class TranslatorManager {
// Helper function to initialize translators for the selected language
private fun initializeTranslators(selectedLanguageCode: String) {
// Set up translator for selected language -> English
- val toEnglishOptions = TranslatorOptions.Builder()
- .setSourceLanguage(selectedLanguageCode)
- .setTargetLanguage(TranslateLanguage.ENGLISH)
- .build()
+ val toEnglishOptions =
+ TranslatorOptions
+ .Builder()
+ .setSourceLanguage(selectedLanguageCode)
+ .setTargetLanguage(TranslateLanguage.ENGLISH)
+ .build()
toEnglishTranslator = Translation.getClient(toEnglishOptions)
// Set up translator for English -> selected language
- val fromEnglishOptions = TranslatorOptions.Builder()
- .setSourceLanguage(TranslateLanguage.ENGLISH)
- .setTargetLanguage(selectedLanguageCode)
- .build()
+ val fromEnglishOptions =
+ TranslatorOptions
+ .Builder()
+ .setSourceLanguage(TranslateLanguage.ENGLISH)
+ .setTargetLanguage(selectedLanguageCode)
+ .build()
fromEnglishTranslator = Translation.getClient(fromEnglishOptions)
}
-
// Use this method to translate from selected language to English
- fun translateToEnglish(text: String, callback: (translatedText: String?) -> Unit) {
- toEnglishTranslator?.translate(text)
+ fun translateToEnglish(
+ text: String,
+ callback: (translatedText: String?) -> Unit,
+ ) {
+ toEnglishTranslator
+ ?.translate(text)
?.addOnSuccessListener { translatedText ->
callback(translatedText)
- }
- ?.addOnFailureListener { e ->
+ }?.addOnFailureListener { e ->
Log.e("Translator", "Translation to English failed: ${e.message}")
callback(null)
}
}
// Use this method to translate from English to the selected language
- fun translateFromEnglish(text: String, callback: (translatedText: String?) -> Unit) {
- fromEnglishTranslator?.translate(text)
+ fun translateFromEnglish(
+ text: String,
+ callback: (translatedText: String?) -> Unit,
+ ) {
+ fromEnglishTranslator
+ ?.translate(text)
?.addOnSuccessListener { translatedText ->
callback(translatedText)
- }
- ?.addOnFailureListener { e ->
+ }?.addOnFailureListener { e ->
Log.e("Translator", "Translation from English failed: ${e.message}")
callback(null)
}
}
- private fun downloadLanguageModel(languageCode: String, onDownloadComplete: (Boolean) -> Unit) {
+ private fun downloadLanguageModel(
+ languageCode: String,
+ onDownloadComplete: (Boolean) -> Unit,
+ ) {
val languageModel = TranslateRemoteModel.Builder(languageCode).build()
val conditions = DownloadConditions.Builder().build()
- modelManager.download(languageModel, conditions)
+ modelManager
+ .download(languageModel, conditions)
.addOnSuccessListener {
Log.d("Translator", "Language model for $languageCode downloaded successfully.")
onDownloadComplete(true)
- }
- .addOnFailureListener { e ->
+ }.addOnFailureListener { e ->
Log.e("Translator", "Failed to download language model for $languageCode: ${e.message}")
onDownloadComplete(false)
}
@@ -122,17 +144,17 @@ class TranslatorManager {
// Delete the previously downloaded language model
private fun deletePreviousLanguageModel(languageCode: String) {
val languageModel = TranslateRemoteModel.Builder(languageCode).build()
- modelManager.deleteDownloadedModel(languageModel)
+ modelManager
+ .deleteDownloadedModel(languageModel)
.addOnSuccessListener {
Log.d("Translator", "Language model for $languageCode deleted successfully.")
- }
- .addOnFailureListener { e ->
+ }.addOnFailureListener { e ->
Log.e("Translator", "Failed to delete language model for $languageCode: ${e.message}")
}
}
- fun closeTranslator(){
+ fun closeTranslator() {
toEnglishTranslator?.close()
fromEnglishTranslator?.close()
}
-}
\ No newline at end of file
+}
diff --git a/app/src/main/java/com/app/assistant/tts/ApiTtsManager.kt b/app/src/main/java/com/app/assistant/tts/ApiTtsManager.kt
new file mode 100644
index 0000000..f1b2367
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/tts/ApiTtsManager.kt
@@ -0,0 +1,76 @@
+package com.app.assistant.tts
+
+import android.content.Context
+import com.app.assistant.config.SpeechConfig
+import com.app.assistant.config.TtsApiProvider
+import com.app.assistant.repository.SettingsRepository
+import okhttp3.OkHttpClient
+
+class ApiTtsManager(
+ private val context: Context,
+ private val settingsRepository: SettingsRepository,
+ private val okHttpClient: OkHttpClient,
+ private val onSpeakingStateChanged: (isSpeaking: Boolean) -> Unit
+) : TtsManager {
+
+ private var edgeTtsApiManager: EdgeTtsApiManager? = null
+ private var googleTtsApiManager: GoogleTtsApiManager? = null
+
+ private fun getEdgeTtsApiManager(): EdgeTtsApiManager {
+ return edgeTtsApiManager ?: EdgeTtsApiManager(
+ context,
+ settingsRepository,
+ okHttpClient,
+ ::handleSpeakingStateChanged
+ ).also { edgeTtsApiManager = it }
+ }
+
+ private fun getGoogleTtsApiManager(): GoogleTtsApiManager {
+ return googleTtsApiManager ?: GoogleTtsApiManager(
+ context,
+ settingsRepository,
+ okHttpClient,
+ ::handleSpeakingStateChanged
+ ).also { googleTtsApiManager = it }
+ }
+
+ private val activeProvider: TtsApiProvider
+ get() = SpeechConfig.ACTIVE_TTS_PROVIDER
+
+ private val currentManager: TtsManager
+ get() = when (activeProvider) {
+ TtsApiProvider.EDGE_TTS -> getEdgeTtsApiManager()
+ TtsApiProvider.GOOGLE_TTS -> getGoogleTtsApiManager()
+ }
+
+ private fun handleSpeakingStateChanged(isSpeaking: Boolean) {
+ onSpeakingStateChanged(isSpeaking)
+ }
+
+ override fun speak(text: String, queueMode: Int) {
+ // Stop the other manager to prevent overlapped audio if provider was changed dynamically
+ when (activeProvider) {
+ TtsApiProvider.EDGE_TTS -> googleTtsApiManager?.stop()
+ TtsApiProvider.GOOGLE_TTS -> edgeTtsApiManager?.stop()
+ }
+ currentManager.speak(text, queueMode)
+ }
+
+ override fun stop() {
+ edgeTtsApiManager?.stop()
+ googleTtsApiManager?.stop()
+ }
+
+ override fun isSpeaking(): Boolean {
+ val manager = when (activeProvider) {
+ TtsApiProvider.EDGE_TTS -> edgeTtsApiManager
+ TtsApiProvider.GOOGLE_TTS -> googleTtsApiManager
+ }
+ return manager?.isSpeaking() ?: false
+ }
+
+ override fun shutdown() {
+ edgeTtsApiManager?.shutdown()
+ googleTtsApiManager?.shutdown()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/tts/EdgeTtsApiManager.kt b/app/src/main/java/com/app/assistant/tts/EdgeTtsApiManager.kt
new file mode 100644
index 0000000..3ad1da9
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/tts/EdgeTtsApiManager.kt
@@ -0,0 +1,656 @@
+package com.app.assistant.tts
+
+import android.content.Context
+import android.media.AudioAttributes
+import android.media.AudioFormat
+import android.media.AudioManager
+import android.media.AudioTrack
+import android.media.MediaCodec
+import android.media.MediaFormat
+import android.util.Log
+import com.app.assistant.config.SpeechConfig
+import com.app.assistant.repository.SettingsRepository
+import kotlinx.coroutines.*
+import kotlinx.coroutines.channels.Channel
+import okhttp3.*
+import okio.ByteString
+import java.io.ByteArrayOutputStream
+import java.security.MessageDigest
+import java.text.SimpleDateFormat
+import java.util.*
+import java.util.concurrent.ConcurrentHashMap
+
+@OptIn(ExperimentalCoroutinesApi::class, ObsoleteCoroutinesApi::class, DelicateCoroutinesApi::class)
+class EdgeTtsApiManager(
+ private val context: Context,
+ private val settingsRepository: SettingsRepository,
+ okHttpClient: OkHttpClient,
+ private val onSpeakingStateChanged: (isSpeaking: Boolean) -> Unit
+) : TtsManager {
+
+ private val ttsDispatcher = newSingleThreadContext("EdgeTtsApiThread")
+ private val job = SupervisorJob()
+ private val scope = CoroutineScope(ttsDispatcher + job)
+
+ private val speechQueue = Channel(Channel.UNLIMITED)
+ private val streamChannel = Channel(Channel.UNLIMITED)
+
+ private val activeStreams = ConcurrentHashMap()
+
+ private var webSocket: WebSocket? = null
+ private var activeWebSocketCall: Call? = null
+
+ private var generatorJob: Job? = null
+ private var playbackJob: Job? = null
+ private var idleTimeoutJob: Job? = null
+
+ private val client = okHttpClient.newBuilder()
+ .connectTimeout(5, java.util.concurrent.TimeUnit.SECONDS)
+ .readTimeout(5, java.util.concurrent.TimeUnit.SECONDS)
+ .writeTimeout(5, java.util.concurrent.TimeUnit.SECONDS)
+ .build()
+
+ @Volatile
+ private var isSpeaking = false
+ @Volatile
+ private var isGenerating = false
+ @Volatile
+ private var isPlaying = false
+
+ private class SentenceStream(
+ val requestId: String,
+ val rawDataChannel: Channel = Channel(Channel.UNLIMITED),
+ val frameChannel: Channel = Channel(Channel.UNLIMITED),
+ val completionDeferred: CompletableDeferred = CompletableDeferred()
+ )
+
+ init {
+ Log.i("EdgeTtsApiManager", "[EdgeTTS] Initialized")
+ startProcessing()
+ }
+
+ private fun startProcessing() {
+ generatorJob?.cancel()
+ playbackJob?.cancel()
+
+ generatorJob = scope.launch {
+ for (text in speechQueue) {
+ isGenerating = true
+ updateSpeakingState()
+
+ try {
+ ensureWebSocketConnected()
+ } catch (e: Exception) {
+ Log.e("EdgeTtsApiManager", "Could not connect to WebSocket, skipping text: $text", e)
+ isGenerating = false
+ updateSpeakingState()
+ continue
+ }
+
+ idleTimeoutJob?.cancel()
+
+ val sentences = splitIntoSentences(text)
+ for (sentence in sentences) {
+ val cleaned = cleanTextForTts(sentence)
+ if (cleaned.isEmpty()) continue
+
+ val requestId = UUID.randomUUID().toString().replace("-", "").lowercase()
+ val stream = SentenceStream(requestId)
+ activeStreams[requestId] = stream
+
+ launchParser(stream)
+
+ try {
+ sendSsmlRequest(requestId, cleaned)
+ streamChannel.send(stream)
+ stream.completionDeferred.await()
+ } catch (e: Exception) {
+ Log.e("EdgeTtsApiManager", "Error in sequential streaming for requestId: $requestId", e)
+ stream.rawDataChannel.close()
+ stream.completionDeferred.complete(Unit)
+ }
+
+ activeStreams.remove(requestId)
+ }
+
+ if (speechQueue.isEmpty) {
+ isGenerating = false
+ updateSpeakingState()
+ resetIdleTimeout()
+ }
+ }
+ }
+
+ playbackJob = scope.launch {
+ for (stream in streamChannel) {
+ isPlaying = true
+ updateSpeakingState()
+
+ playStream(stream)
+
+ if (streamChannel.isEmpty && !isGenerating && speechQueue.isEmpty) {
+ isPlaying = false
+ updateSpeakingState()
+ }
+ }
+ }
+ }
+
+ private suspend fun playStream(stream: SentenceStream) {
+ withContext(Dispatchers.Default) {
+ val sampleRate = 24000
+ val channelConfig = AudioFormat.CHANNEL_OUT_MONO
+ val audioFormat = AudioFormat.ENCODING_PCM_16BIT
+ val minBufferSize = AudioTrack.getMinBufferSize(sampleRate, channelConfig, audioFormat)
+
+ val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
+ val useVoiceCall = audioManager.isBluetoothScoOn ||
+ audioManager.mode == AudioManager.MODE_IN_CALL ||
+ audioManager.mode == AudioManager.MODE_IN_COMMUNICATION
+
+ val audioTrack = AudioTrack.Builder()
+ .setAudioAttributes(
+ AudioAttributes.Builder()
+ .setUsage(if (useVoiceCall) AudioAttributes.USAGE_VOICE_COMMUNICATION else AudioAttributes.USAGE_MEDIA)
+ .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
+ .build()
+ )
+ .setAudioFormat(
+ AudioFormat.Builder()
+ .setEncoding(audioFormat)
+ .setSampleRate(sampleRate)
+ .setChannelMask(channelConfig)
+ .build()
+ )
+ .setBufferSizeInBytes(Math.max(minBufferSize, 1024 * 16))
+ .setTransferMode(AudioTrack.MODE_STREAM)
+ .build()
+
+ var codec: MediaCodec? = null
+ try {
+ audioTrack.play()
+
+ val mediaFormat = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_MPEG, sampleRate, 1)
+ codec = MediaCodec.createDecoderByType(MediaFormat.MIMETYPE_AUDIO_MPEG).apply {
+ configure(mediaFormat, null, null, 0)
+ start()
+ }
+
+ var decodingActive = true
+ var hasMoreInput = true
+ var presentationTimeUs = 0L
+ val frameIterator = stream.frameChannel.iterator()
+
+ while (decodingActive && coroutineContext.isActive) {
+ var didWork = false
+
+ if (hasMoreInput) {
+ val inputIndex = codec.dequeueInputBuffer(0)
+ if (inputIndex >= 0) {
+ val inputBuffer = codec.getInputBuffer(inputIndex)
+ if (inputBuffer != null) {
+ inputBuffer.clear()
+ if (frameIterator.hasNext()) {
+ val frame = frameIterator.next()
+ inputBuffer.put(frame)
+ codec.queueInputBuffer(inputIndex, 0, frame.size, presentationTimeUs, 0)
+ presentationTimeUs += (frame.size * 8000000L) / 48000L
+ didWork = true
+ } else {
+ if (stream.frameChannel.isClosedForReceive) {
+ codec.queueInputBuffer(inputIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM)
+ hasMoreInput = false
+ didWork = true
+ }
+ }
+ }
+ }
+ }
+
+ val bufferInfo = MediaCodec.BufferInfo()
+ val outputIndex = codec.dequeueOutputBuffer(bufferInfo, 0)
+ if (outputIndex >= 0) {
+ val outputBuffer = codec.getOutputBuffer(outputIndex)
+ if (outputBuffer != null && bufferInfo.size > 0) {
+ val pcmData = ByteArray(bufferInfo.size)
+ outputBuffer.get(pcmData)
+ outputBuffer.clear()
+ audioTrack.write(pcmData, 0, pcmData.size)
+ }
+ codec.releaseOutputBuffer(outputIndex, false)
+
+ if ((bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
+ decodingActive = false
+ }
+ didWork = true
+ }
+
+ if (!didWork) {
+ delay(5)
+ }
+ }
+
+ if (coroutineContext.isActive) {
+ delay(150)
+ }
+
+ } catch (e: Exception) {
+ Log.e("EdgeTtsApiManager", "Error decoding/playing stream", e)
+ } finally {
+ try {
+ audioTrack.stop()
+ audioTrack.flush()
+ audioTrack.release()
+ } catch (e: Exception) {}
+ try {
+ codec?.stop()
+ codec?.release()
+ } catch (e: Exception) {}
+ }
+ Unit
+ }
+ }
+
+ private fun launchParser(stream: SentenceStream): Job {
+ return scope.launch(Dispatchers.Default) {
+ val parserBuffer = ByteArrayOutputStream()
+ try {
+ for (bytes in stream.rawDataChannel) {
+ parserBuffer.write(bytes)
+ var bufferBytes = parserBuffer.toByteArray()
+ var offset = 0
+ while (offset + 4 <= bufferBytes.size) {
+ val b0 = bufferBytes[offset].toInt() and 0xFF
+ val b1 = bufferBytes[offset + 1].toInt() and 0xFF
+ if (b0 == 0xFF && (b1 and 0xE0) == 0xE0) {
+ val version = (b1 shr 3) and 0x03
+ val layer = (b1 shr 1) and 0x03
+ val bitrateIndex = (bufferBytes[offset + 2].toInt() and 0xFF) shr 4
+ val srIndex = (bufferBytes[offset + 2].toInt() and 0xFF shr 2) and 0x03
+ val padding = (bufferBytes[offset + 2].toInt() and 0xFF shr 1) and 0x01
+
+ val mpeg1Bitrates = intArrayOf(0, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 0)
+ val mpeg2Bitrates = intArrayOf(0, 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160, 0)
+ val sampleRates = arrayOf(
+ intArrayOf(11025, 12000, 8000),
+ intArrayOf(0, 0, 0),
+ intArrayOf(22050, 24000, 16000),
+ intArrayOf(44100, 48000, 32000)
+ )
+
+ if (version < sampleRates.size && srIndex < sampleRates[version].size) {
+ val bitrate = (if (version == 3) mpeg1Bitrates[bitrateIndex] else mpeg2Bitrates[bitrateIndex]) * 1000
+ val sampleRate = sampleRates[version][srIndex]
+
+ if (bitrate > 0 && sampleRate > 0 && layer == 1) {
+ val coeff = if (version == 3) 144 else 72
+ val frameSize = (coeff * bitrate) / sampleRate + padding
+
+ if (offset + frameSize <= bufferBytes.size) {
+ val frame = bufferBytes.copyOfRange(offset, offset + frameSize)
+ stream.frameChannel.send(frame)
+ offset += frameSize
+ continue
+ } else {
+ break
+ }
+ }
+ }
+ }
+ offset++
+ }
+
+ val temp = parserBuffer.toByteArray()
+ parserBuffer.reset()
+ if (offset < temp.size) {
+ parserBuffer.write(temp, offset, temp.size - offset)
+ }
+ }
+ } catch (e: Exception) {
+ Log.e("EdgeTtsApiManager", "Error parsing MP3 frames", e)
+ } finally {
+ stream.frameChannel.close()
+ }
+ }
+ }
+
+ private suspend fun ensureWebSocketConnected() {
+ if (webSocket != null) return
+
+ val connectionId = UUID.randomUUID().toString().replace("-", "").uppercase()
+ val secMsGec = generateSecMsGec()
+ val url = "${SpeechConfig.EdgeTts.WSS_URL}" +
+ "?Ocp-Apim-Subscription-Key=${SpeechConfig.EdgeTts.getSubscriptionKey(settingsRepository)}" +
+ "&ConnectionId=$connectionId" +
+ "&Sec-MS-GEC=$secMsGec" +
+ "&Sec-MS-GEC-Version=1-130.0.2849.68"
+
+ val request = Request.Builder()
+ .url(url)
+ .addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0")
+ .addHeader("Origin", "chrome-extension://jdiccldimpdaibmpdkjnbmckianbfold")
+ .addHeader("Pragma", "no-cache")
+ .addHeader("Cache-Control", "no-cache")
+ .build()
+
+ val connectDeferred = CompletableDeferred()
+ val listener = object : WebSocketListener() {
+ override fun onOpen(ws: WebSocket, response: Response) {
+ Log.d("EdgeTtsApiManager", "WebSocket connected successfully")
+ connectDeferred.complete(ws)
+ }
+
+ override fun onMessage(ws: WebSocket, text: String) {
+ handleTextMessage(text)
+ }
+
+ override fun onMessage(ws: WebSocket, bytes: ByteString) {
+ handleBinaryMessage(bytes.toByteArray())
+ }
+
+ override fun onClosing(ws: WebSocket, code: Int, reason: String) {
+ ws.close(1000, null)
+ }
+
+ override fun onClosed(ws: WebSocket, code: Int, reason: String) {
+ Log.d("EdgeTtsApiManager", "WebSocket closed")
+ if (webSocket == ws) {
+ webSocket = null
+ }
+ }
+
+ override fun onFailure(ws: WebSocket, t: Throwable, response: Response?) {
+ Log.e("EdgeTtsApiManager", "WebSocket failure: ${t.message}", t)
+ if (connectDeferred.isActive) {
+ connectDeferred.completeExceptionally(t)
+ }
+ if (webSocket == ws) {
+ webSocket = null
+ }
+ for (stream in activeStreams.values) {
+ stream.rawDataChannel.close()
+ stream.completionDeferred.complete(Unit)
+ }
+ activeStreams.clear()
+ }
+ }
+
+ activeWebSocketCall?.cancel()
+
+ try {
+ webSocket = client.newWebSocket(request, listener)
+ withTimeout(5000L) {
+ connectDeferred.await()
+ }
+ } catch (e: Exception) {
+ Log.e("EdgeTtsApiManager", "Failed to connect WebSocket", e)
+ webSocket = null
+ throw e
+ }
+ }
+
+ private fun handleTextMessage(text: String) {
+ val lines = text.lines()
+ var reqId: String? = null
+ var isTurnEnd = false
+ for (line in lines) {
+ val trimmed = line.trim()
+ if (trimmed.startsWith("X-RequestId:", ignoreCase = true)) {
+ reqId = trimmed.substringAfter(":").trim()
+ }
+ if (trimmed.startsWith("Path:", ignoreCase = true)) {
+ val path = trimmed.substringAfter(":").trim()
+ if (path.equals("turn.end", ignoreCase = true)) {
+ isTurnEnd = true
+ }
+ }
+ }
+
+ if (reqId != null && isTurnEnd) {
+ val stream = activeStreams[reqId]
+ if (stream != null) {
+ stream.rawDataChannel.close()
+ stream.completionDeferred.complete(Unit)
+ }
+ }
+ }
+
+ private fun handleBinaryMessage(bytesArray: ByteArray) {
+ if (bytesArray.size < 2) return
+ val headerLength = ((bytesArray[0].toInt() and 0xFF) shl 8) or (bytesArray[1].toInt() and 0xFF)
+ if (headerLength + 2 <= bytesArray.size) {
+ val headerStr = String(bytesArray, 2, headerLength, Charsets.UTF_8)
+ val lines = headerStr.lines()
+ var reqId: String? = null
+ var isAudio = false
+ for (line in lines) {
+ val trimmed = line.trim()
+ if (trimmed.startsWith("X-RequestId:", ignoreCase = true)) {
+ reqId = trimmed.substringAfter(":").trim()
+ }
+ if (trimmed.startsWith("Path:", ignoreCase = true)) {
+ val path = trimmed.substringAfter(":").trim()
+ if (path.equals("audio", ignoreCase = true)) {
+ isAudio = true
+ }
+ }
+ }
+
+ if (reqId != null && isAudio) {
+ val stream = activeStreams[reqId]
+ if (stream != null) {
+ val audioOffset = 2 + headerLength
+ val audioData = bytesArray.copyOfRange(audioOffset, bytesArray.size)
+ if (audioData.isNotEmpty()) {
+ stream.rawDataChannel.trySend(audioData)
+ }
+ }
+ }
+ }
+ }
+
+ private fun sendSsmlRequest(requestId: String, text: String) {
+ val ws = webSocket ?: return
+ val dateStr = getFormattedDate()
+
+ val configMsg = "X-Timestamp:$dateStr\r\n" +
+ "Content-Type:application/json; charset=utf-8\r\n" +
+ "Path:speech.config\r\n\r\n" +
+ "{\"context\":{\"synthesis\":{\"audio\":{\"metadataoptions\":{\"sentenceBoundaryEnabled\":\"true\",\"wordBoundaryEnabled\":\"true\"},\"outputFormat\":\"${SpeechConfig.EdgeTts.OUTPUT_FORMAT}\"}}}}"
+ ws.send(configMsg)
+
+ val escapedText = escapeXml(text)
+ val ssmlMsg = "X-RequestId:$requestId\r\n" +
+ "Content-Type:application/ssml+xml\r\n" +
+ "X-Timestamp:$dateStr\r\n" +
+ "Path:ssml\r\n\r\n" +
+ "" +
+ "" +
+ "" +
+ escapedText +
+ " "
+
+ ws.send(ssmlMsg)
+ }
+
+ private fun getFormattedDate(): String {
+ val sdf = SimpleDateFormat("EEE MMM dd yyyy HH:mm:ss 'GMT+0000 (Coordinated Universal Time)'", Locale.US)
+ sdf.timeZone = TimeZone.getTimeZone("UTC")
+ return sdf.format(Date())
+ }
+
+ private fun escapeXml(text: String): String {
+ return text
+ .replace("&", "&")
+ .replace("<", "<")
+ .replace(">", ">")
+ .replace("\"", """)
+ .replace("'", "'")
+ }
+
+ private fun generateSecMsGec(): String {
+ val unixTime = System.currentTimeMillis() / 1000
+ var ticks = unixTime
+ ticks -= ticks % 300
+ val winTicks = (ticks + 11644473600L) * 10000000L
+ val strToHash = "${winTicks}${SpeechConfig.EdgeTts.getSubscriptionKey(settingsRepository)}"
+ val md = MessageDigest.getInstance("SHA-256")
+ val digest = md.digest(strToHash.toByteArray(Charsets.UTF_8))
+ return digest.joinToString("") { "%02X".format(it) }
+ }
+
+ private fun cleanTextForTts(text: String): String {
+ return text
+ .replace("—", " - ")
+ .replace("–", " - ")
+ .replace("“", "")
+ .replace("”", "")
+ .replace("\"", "")
+ .replace("‘", "'")
+ .replace("’", "'")
+ .trim()
+ }
+
+ private fun splitIntoSentences(text: String): List {
+ val sentences = mutableListOf()
+ val currentSentence = StringBuilder()
+ val len = text.length
+ var i = 0
+ while (i < len) {
+ val c = text[i]
+ currentSentence.append(c)
+
+ if (c == '\n' || c == '\r') {
+ val sentenceStr = currentSentence.toString().trim()
+ if (sentenceStr.isNotEmpty()) {
+ sentences.add(sentenceStr)
+ }
+ currentSentence.clear()
+ i++
+ continue
+ }
+
+ if (c == '.' || c == '?' || c == '!' || c == ':' || c == ';') {
+ var isBoundary = false
+ if (i + 1 == len) {
+ isBoundary = true
+ } else {
+ val nextChar = text[i + 1]
+ if (nextChar.isWhitespace()) {
+ isBoundary = true
+ }
+ }
+
+ if (isBoundary) {
+ if (c == '.' && isAbbreviationOrDecimal(text, i)) {
+ // abbreviation
+ } else {
+ val sentenceStr = currentSentence.toString().trim()
+ if (sentenceStr.isNotEmpty()) {
+ sentences.add(sentenceStr)
+ }
+ currentSentence.clear()
+ }
+ }
+ }
+ i++
+ }
+
+ val remaining = currentSentence.toString().trim()
+ if (remaining.isNotEmpty()) {
+ sentences.add(remaining)
+ }
+
+ return sentences
+ }
+
+ private fun isAbbreviationOrDecimal(text: String, dotIndex: Int): Boolean {
+ if (dotIndex > 0 && dotIndex + 1 < text.length) {
+ if (text[dotIndex - 1].isDigit() && text[dotIndex + 1].isDigit()) {
+ return true
+ }
+ }
+ var start = dotIndex - 1
+ while (start >= 0 && text[start].isLetter()) {
+ start--
+ }
+ val word = text.substring(start + 1, dotIndex).lowercase()
+ val abbreviations = setOf(
+ "mr", "mrs", "ms", "dr", "prof", "sr", "jr", "eg", "ie", "vs", "etc", "st", "co",
+ "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p",
+ "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"
+ )
+ return word in abbreviations
+ }
+
+ private fun resetIdleTimeout() {
+ idleTimeoutJob?.cancel()
+ idleTimeoutJob = scope.launch {
+ delay(10000L)
+ closeWebSocketConnection()
+ }
+ }
+
+ private fun closeWebSocketConnection() {
+ webSocket?.close(1000, "Idle timeout")
+ webSocket = null
+ }
+
+ private fun updateSpeakingState() {
+ val shouldBeSpeaking = isGenerating || isPlaying || !speechQueue.isEmpty || !streamChannel.isEmpty
+ if (shouldBeSpeaking != isSpeaking) {
+ isSpeaking = shouldBeSpeaking
+ scope.launch(Dispatchers.Main) {
+ onSpeakingStateChanged(shouldBeSpeaking)
+ }
+ }
+ }
+
+ override fun speak(text: String, queueMode: Int) {
+ scope.launch {
+ if (queueMode == TtsManager.QUEUE_FLUSH) {
+ stopPlaybackOnly()
+ }
+ speechQueue.send(text)
+ updateSpeakingState()
+ }
+ }
+
+ private fun stopPlaybackOnly() {
+ Log.i("EdgeTtsApiManager", "[EdgeTTS] Playback stopped/cleared")
+ while (!speechQueue.isEmpty) {
+ speechQueue.tryReceive()
+ }
+ while (!streamChannel.isEmpty) {
+ val stream = streamChannel.tryReceive().getOrNull()
+ stream?.rawDataChannel?.close()
+ stream?.completionDeferred?.complete(Unit)
+ }
+ for (stream in activeStreams.values) {
+ stream.rawDataChannel.close()
+ stream.completionDeferred.complete(Unit)
+ }
+ activeStreams.clear()
+
+ startProcessing()
+
+ isGenerating = false
+ isPlaying = false
+ updateSpeakingState()
+ }
+
+ override fun stop() {
+ stopPlaybackOnly()
+ }
+
+ override fun isSpeaking(): Boolean {
+ return isSpeaking
+ }
+
+ override fun shutdown() {
+ Log.i("EdgeTtsApiManager", "[EdgeTTS] Shutdown")
+ job.cancel()
+ closeWebSocketConnection()
+ ttsDispatcher.close()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/tts/GoogleTtsApiManager.kt b/app/src/main/java/com/app/assistant/tts/GoogleTtsApiManager.kt
new file mode 100644
index 0000000..cbccb1c
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/tts/GoogleTtsApiManager.kt
@@ -0,0 +1,488 @@
+package com.app.assistant.tts
+
+import android.content.Context
+import android.media.AudioAttributes
+import android.media.AudioFormat
+import android.media.AudioManager
+import android.media.AudioTrack
+import android.media.MediaCodec
+import android.media.MediaFormat
+import android.util.Log
+import com.app.assistant.config.SpeechConfig
+import com.app.assistant.repository.SettingsRepository
+import kotlinx.coroutines.*
+import kotlinx.coroutines.channels.Channel
+import okhttp3.*
+import okhttp3.MediaType.Companion.toMediaType
+import okhttp3.RequestBody.Companion.toRequestBody
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.buildJsonObject
+import kotlinx.serialization.json.put
+import kotlinx.serialization.json.putJsonObject
+import kotlinx.serialization.json.jsonObject
+import kotlinx.serialization.json.jsonPrimitive
+import java.io.ByteArrayOutputStream
+import java.util.*
+import java.util.concurrent.ConcurrentHashMap
+
+@OptIn(ExperimentalCoroutinesApi::class, ObsoleteCoroutinesApi::class, DelicateCoroutinesApi::class)
+class GoogleTtsApiManager(
+ private val context: Context,
+ private val settingsRepository: SettingsRepository,
+ okHttpClient: OkHttpClient,
+ private val onSpeakingStateChanged: (isSpeaking: Boolean) -> Unit
+) : TtsManager {
+
+ private val ttsDispatcher = newSingleThreadContext("GoogleTtsApiThread")
+ private val job = SupervisorJob()
+ private val scope = CoroutineScope(ttsDispatcher + job)
+
+ private val speechQueue = Channel(Channel.UNLIMITED)
+ private val streamChannel = Channel(Channel.UNLIMITED)
+
+ private val activeStreams = ConcurrentHashMap()
+
+ private var generatorJob: Job? = null
+ private var playbackJob: Job? = null
+
+ private val client = okHttpClient.newBuilder()
+ .connectTimeout(10, java.util.concurrent.TimeUnit.SECONDS)
+ .readTimeout(10, java.util.concurrent.TimeUnit.SECONDS)
+ .writeTimeout(10, java.util.concurrent.TimeUnit.SECONDS)
+ .build()
+
+ @Volatile
+ private var isSpeaking = false
+ @Volatile
+ private var isGenerating = false
+ @Volatile
+ private var isPlaying = false
+
+ private class SentenceStream(
+ val requestId: String,
+ val rawDataChannel: Channel = Channel(Channel.UNLIMITED),
+ val frameChannel: Channel = Channel(Channel.UNLIMITED)
+ )
+
+ init {
+ Log.i("GoogleTtsApiManager", "[GoogleTTS] Initialized")
+ startProcessing()
+ }
+
+ private fun startProcessing() {
+ generatorJob?.cancel()
+ playbackJob?.cancel()
+
+ generatorJob = scope.launch {
+ for (text in speechQueue) {
+ isGenerating = true
+ updateSpeakingState()
+
+ val sentences = splitIntoSentences(text)
+ for (sentence in sentences) {
+ val cleaned = cleanTextForTts(sentence)
+ if (cleaned.isEmpty()) continue
+
+ val requestId = UUID.randomUUID().toString().replace("-", "").lowercase()
+ val stream = SentenceStream(requestId)
+ activeStreams[requestId] = stream
+
+ launchParser(stream)
+
+ try {
+ val audioBytes = fetchGoogleTts(cleaned)
+ stream.rawDataChannel.send(audioBytes)
+ stream.rawDataChannel.close()
+ streamChannel.send(stream)
+ } catch (e: Exception) {
+ Log.e("GoogleTtsApiManager", "Error fetching Google TTS for sentence: $cleaned", e)
+ stream.rawDataChannel.close()
+ }
+
+ activeStreams.remove(requestId)
+ }
+
+ if (speechQueue.isEmpty) {
+ isGenerating = false
+ updateSpeakingState()
+ }
+ }
+ }
+
+ playbackJob = scope.launch {
+ for (stream in streamChannel) {
+ isPlaying = true
+ updateSpeakingState()
+
+ playStream(stream)
+
+ if (streamChannel.isEmpty && !isGenerating && speechQueue.isEmpty) {
+ isPlaying = false
+ updateSpeakingState()
+ }
+ }
+ }
+ }
+
+ private suspend fun fetchGoogleTts(text: String): ByteArray = withContext(Dispatchers.IO) {
+ val apiKey = SpeechConfig.GoogleTts.getApiKey(settingsRepository)
+ if (apiKey.isEmpty()) {
+ throw Exception("Google Cloud TTS API key is empty.")
+ }
+ val url = "${SpeechConfig.GoogleTts.BASE_URL}?key=$apiKey"
+
+ val voiceName = SpeechConfig.GoogleTts.VOICE
+ val parts = voiceName.split("-")
+ val languageCode = if (parts.size >= 2) "${parts[0]}-${parts[1]}" else "en-US"
+
+ val jsonBodyElement = buildJsonObject {
+ putJsonObject("input") {
+ put("text", text)
+ }
+ putJsonObject("voice") {
+ put("languageCode", languageCode)
+ put("name", voiceName)
+ }
+ putJsonObject("audioConfig") {
+ put("audioEncoding", SpeechConfig.GoogleTts.AUDIO_ENCODING)
+ put("sampleRateHertz", 24000)
+ }
+ }
+ val jsonBody = jsonBodyElement.toString()
+
+ val requestBody = jsonBody.toRequestBody("application/json; charset=utf-8".toMediaType())
+ val request = Request.Builder()
+ .url(url)
+ .post(requestBody)
+ .build()
+
+ client.newCall(request).execute().use { response ->
+ if (!response.isSuccessful) {
+ val errBody = response.body?.string() ?: ""
+ throw Exception("Google Cloud TTS API call failed: ${response.code} - ${response.message}. Body: $errBody")
+ }
+ val bodyStr = response.body?.string() ?: throw Exception("Empty response body from Google TTS")
+ val jsonElement = Json.parseToJsonElement(bodyStr)
+ val audioContentBase64 = jsonElement.jsonObject["audioContent"]?.jsonPrimitive?.content
+ ?: throw Exception("No audioContent field in response JSON")
+ android.util.Base64.decode(audioContentBase64, android.util.Base64.DEFAULT)
+ }
+ }
+
+ private suspend fun playStream(stream: SentenceStream) {
+ withContext(Dispatchers.Default) {
+ val sampleRate = 24000
+ val channelConfig = AudioFormat.CHANNEL_OUT_MONO
+ val audioFormat = AudioFormat.ENCODING_PCM_16BIT
+ val minBufferSize = AudioTrack.getMinBufferSize(sampleRate, channelConfig, audioFormat)
+
+ val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
+ val useVoiceCall = audioManager.isBluetoothScoOn ||
+ audioManager.mode == AudioManager.MODE_IN_CALL ||
+ audioManager.mode == AudioManager.MODE_IN_COMMUNICATION
+
+ val audioTrack = AudioTrack.Builder()
+ .setAudioAttributes(
+ AudioAttributes.Builder()
+ .setUsage(if (useVoiceCall) AudioAttributes.USAGE_VOICE_COMMUNICATION else AudioAttributes.USAGE_MEDIA)
+ .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
+ .build()
+ )
+ .setAudioFormat(
+ AudioFormat.Builder()
+ .setEncoding(audioFormat)
+ .setSampleRate(sampleRate)
+ .setChannelMask(channelConfig)
+ .build()
+ )
+ .setBufferSizeInBytes(Math.max(minBufferSize, 1024 * 16))
+ .setTransferMode(AudioTrack.MODE_STREAM)
+ .build()
+
+ var codec: MediaCodec? = null
+ try {
+ audioTrack.play()
+
+ val mediaFormat = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_MPEG, sampleRate, 1)
+ codec = MediaCodec.createDecoderByType(MediaFormat.MIMETYPE_AUDIO_MPEG).apply {
+ configure(mediaFormat, null, null, 0)
+ start()
+ }
+
+ var decodingActive = true
+ var hasMoreInput = true
+ var presentationTimeUs = 0L
+ val frameIterator = stream.frameChannel.iterator()
+
+ while (decodingActive && coroutineContext.isActive) {
+ var didWork = false
+
+ if (hasMoreInput) {
+ val inputIndex = codec.dequeueInputBuffer(0)
+ if (inputIndex >= 0) {
+ val inputBuffer = codec.getInputBuffer(inputIndex)
+ if (inputBuffer != null) {
+ inputBuffer.clear()
+ if (frameIterator.hasNext()) {
+ val frame = frameIterator.next()
+ inputBuffer.put(frame)
+ codec.queueInputBuffer(inputIndex, 0, frame.size, presentationTimeUs, 0)
+ presentationTimeUs += (frame.size * 8000000L) / 48000L
+ didWork = true
+ } else {
+ if (stream.frameChannel.isClosedForReceive) {
+ codec.queueInputBuffer(inputIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM)
+ hasMoreInput = false
+ didWork = true
+ }
+ }
+ }
+ }
+ }
+
+ val bufferInfo = MediaCodec.BufferInfo()
+ val outputIndex = codec.dequeueOutputBuffer(bufferInfo, 0)
+ if (outputIndex >= 0) {
+ val outputBuffer = codec.getOutputBuffer(outputIndex)
+ if (outputBuffer != null && bufferInfo.size > 0) {
+ val pcmData = ByteArray(bufferInfo.size)
+ outputBuffer.get(pcmData)
+ outputBuffer.clear()
+ audioTrack.write(pcmData, 0, pcmData.size)
+ }
+ codec.releaseOutputBuffer(outputIndex, false)
+
+ if ((bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
+ decodingActive = false
+ }
+ didWork = true
+ }
+
+ if (!didWork) {
+ delay(5)
+ }
+ }
+
+ if (coroutineContext.isActive) {
+ delay(150)
+ }
+
+ } catch (e: Exception) {
+ Log.e("GoogleTtsApiManager", "Error decoding/playing stream", e)
+ } finally {
+ try {
+ audioTrack.stop()
+ audioTrack.flush()
+ audioTrack.release()
+ } catch (e: Exception) {}
+ try {
+ codec?.stop()
+ codec?.release()
+ } catch (e: Exception) {}
+ }
+ Unit
+ }
+ }
+
+ private fun launchParser(stream: SentenceStream): Job {
+ return scope.launch(Dispatchers.Default) {
+ val parserBuffer = ByteArrayOutputStream()
+ try {
+ for (bytes in stream.rawDataChannel) {
+ parserBuffer.write(bytes)
+ var bufferBytes = parserBuffer.toByteArray()
+ var offset = 0
+ while (offset + 4 <= bufferBytes.size) {
+ val b0 = bufferBytes[offset].toInt() and 0xFF
+ val b1 = bufferBytes[offset + 1].toInt() and 0xFF
+ if (b0 == 0xFF && (b1 and 0xE0) == 0xE0) {
+ val version = (b1 shr 3) and 0x03
+ val layer = (b1 shr 1) and 0x03
+ val bitrateIndex = (bufferBytes[offset + 2].toInt() and 0xFF) shr 4
+ val srIndex = (bufferBytes[offset + 2].toInt() and 0xFF shr 2) and 0x03
+ val padding = (bufferBytes[offset + 2].toInt() and 0xFF shr 1) and 0x01
+
+ val mpeg1Bitrates = intArrayOf(0, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 0)
+ val mpeg2Bitrates = intArrayOf(0, 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160, 0)
+ val sampleRates = arrayOf(
+ intArrayOf(11025, 12000, 8000),
+ intArrayOf(0, 0, 0),
+ intArrayOf(22050, 24000, 16000),
+ intArrayOf(44100, 48000, 32000)
+ )
+
+ if (version < sampleRates.size && srIndex < sampleRates[version].size) {
+ val bitrate = (if (version == 3) mpeg1Bitrates[bitrateIndex] else mpeg2Bitrates[bitrateIndex]) * 1000
+ val sampleRate = sampleRates[version][srIndex]
+
+ if (bitrate > 0 && sampleRate > 0 && layer == 1) {
+ val coeff = if (version == 3) 144 else 72
+ val frameSize = (coeff * bitrate) / sampleRate + padding
+
+ if (offset + frameSize <= bufferBytes.size) {
+ val frame = bufferBytes.copyOfRange(offset, offset + frameSize)
+ stream.frameChannel.send(frame)
+ offset += frameSize
+ continue
+ } else {
+ break
+ }
+ }
+ }
+ }
+ offset++
+ }
+
+ val temp = parserBuffer.toByteArray()
+ parserBuffer.reset()
+ if (offset < temp.size) {
+ parserBuffer.write(temp, offset, temp.size - offset)
+ }
+ }
+ } catch (e: Exception) {
+ Log.e("GoogleTtsApiManager", "Error parsing MP3 frames", e)
+ } finally {
+ stream.frameChannel.close()
+ }
+ }
+ }
+
+ private fun cleanTextForTts(text: String): String {
+ return text
+ .replace("—", " - ")
+ .replace("–", " - ")
+ .replace("“", "")
+ .replace("”", "")
+ .replace("\"", "")
+ .replace("‘", "'")
+ .replace("’", "'")
+ .trim()
+ }
+
+ private fun splitIntoSentences(text: String): List {
+ val sentences = mutableListOf()
+ val currentSentence = StringBuilder()
+ val len = text.length
+ var i = 0
+ while (i < len) {
+ val c = text[i]
+ currentSentence.append(c)
+
+ if (c == '\n' || c == '\r') {
+ val sentenceStr = currentSentence.toString().trim()
+ if (sentenceStr.isNotEmpty()) {
+ sentences.add(sentenceStr)
+ }
+ currentSentence.clear()
+ i++
+ continue
+ }
+
+ if (c == '.' || c == '?' || c == '!' || c == ':' || c == ';') {
+ var isBoundary = false
+ if (i + 1 == len) {
+ isBoundary = true
+ } else {
+ val nextChar = text[i + 1]
+ if (nextChar.isWhitespace()) {
+ isBoundary = true
+ }
+ }
+
+ if (isBoundary) {
+ if (c == '.' && isAbbreviationOrDecimal(text, i)) {
+ // abbreviation
+ } else {
+ val sentenceStr = currentSentence.toString().trim()
+ if (sentenceStr.isNotEmpty()) {
+ sentences.add(sentenceStr)
+ }
+ currentSentence.clear()
+ }
+ }
+ }
+ i++
+ }
+
+ val remaining = currentSentence.toString().trim()
+ if (remaining.isNotEmpty()) {
+ sentences.add(remaining)
+ }
+
+ return sentences
+ }
+
+ private fun isAbbreviationOrDecimal(text: String, dotIndex: Int): Boolean {
+ if (dotIndex > 0 && dotIndex + 1 < text.length) {
+ if (text[dotIndex - 1].isDigit() && text[dotIndex + 1].isDigit()) {
+ return true
+ }
+ }
+ var start = dotIndex - 1
+ while (start >= 0 && text[start].isLetter()) {
+ start--
+ }
+ val word = text.substring(start + 1, dotIndex).lowercase()
+ val abbreviations = setOf(
+ "mr", "mrs", "ms", "dr", "prof", "sr", "jr", "eg", "ie", "vs", "etc", "st", "co",
+ "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p",
+ "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"
+ )
+ return word in abbreviations
+ }
+
+ private fun updateSpeakingState() {
+ val shouldBeSpeaking = isGenerating || isPlaying || !speechQueue.isEmpty || !streamChannel.isEmpty
+ if (shouldBeSpeaking != isSpeaking) {
+ isSpeaking = shouldBeSpeaking
+ scope.launch(Dispatchers.Main) {
+ onSpeakingStateChanged(shouldBeSpeaking)
+ }
+ }
+ }
+
+ override fun speak(text: String, queueMode: Int) {
+ scope.launch {
+ if (queueMode == TtsManager.QUEUE_FLUSH) {
+ stopPlaybackOnly()
+ }
+ speechQueue.send(text)
+ updateSpeakingState()
+ }
+ }
+
+ private fun stopPlaybackOnly() {
+ Log.i("GoogleTtsApiManager", "[GoogleTTS] Playback stopped/cleared")
+ while (!speechQueue.isEmpty) {
+ speechQueue.tryReceive()
+ }
+ while (!streamChannel.isEmpty) {
+ val stream = streamChannel.tryReceive().getOrNull()
+ stream?.rawDataChannel?.close()
+ }
+ for (stream in activeStreams.values) {
+ stream.rawDataChannel.close()
+ }
+ activeStreams.clear()
+
+ startProcessing()
+
+ isGenerating = false
+ isPlaying = false
+ updateSpeakingState()
+ }
+
+ override fun stop() {
+ stopPlaybackOnly()
+ }
+
+ override fun isSpeaking(): Boolean {
+ return isSpeaking
+ }
+
+ override fun shutdown() {
+ Log.i("GoogleTtsApiManager", "[GoogleTTS] Shutdown")
+ job.cancel()
+ ttsDispatcher.close()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/tts/NativeTtsManager.kt b/app/src/main/java/com/app/assistant/tts/NativeTtsManager.kt
new file mode 100644
index 0000000..7bd5044
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/tts/NativeTtsManager.kt
@@ -0,0 +1,163 @@
+package com.app.assistant.tts
+
+import android.content.Context
+import android.media.AudioAttributes
+import android.media.AudioManager
+import android.speech.tts.TextToSpeech
+import android.speech.tts.UtteranceProgressListener
+import android.util.Log
+import java.util.Collections
+import java.util.concurrent.ConcurrentHashMap
+
+class NativeTtsManager(
+ private val context: Context,
+ private val onSpeakingStateChanged: (isSpeaking: Boolean) -> Unit
+) : TtsManager {
+ private var textToSpeech: TextToSpeech? = null
+ private var isInitialized = false
+
+ // Track active/queued utterances to prevent premature transition to LISTENING state
+ private val activeUtterances = Collections.newSetFromMap(ConcurrentHashMap())
+
+ // Buffer incoming speak requests while the platform TTS engine is initializing
+ private val pendingUtterances = Collections.synchronizedList(mutableListOf>())
+
+ init {
+ textToSpeech = TextToSpeech(context.applicationContext) { status ->
+ if (status == TextToSpeech.SUCCESS) {
+ isInitialized = true
+ Log.d("NativeTtsManager", "Initialization Success")
+ textToSpeech?.setOnUtteranceProgressListener(object : UtteranceProgressListener() {
+ override fun onStart(utteranceId: String?) {
+ onSpeakingStateChanged(true)
+ }
+
+ override fun onDone(utteranceId: String?) {
+ utteranceId?.let { activeUtterances.remove(it) }
+ if (activeUtterances.isEmpty()) {
+ onSpeakingStateChanged(false)
+ }
+ }
+
+ @Deprecated("Deprecated in Java")
+ override fun onError(utteranceId: String?) {
+ utteranceId?.let { activeUtterances.remove(it) }
+ if (activeUtterances.isEmpty()) {
+ onSpeakingStateChanged(false)
+ }
+ }
+
+ override fun onStop(utteranceId: String?, interrupted: Boolean) {
+ utteranceId?.let { activeUtterances.remove(it) }
+ if (activeUtterances.isEmpty()) {
+ onSpeakingStateChanged(false)
+ }
+ }
+ })
+
+ // Flush any buffered utterances now that the TTS service is bound
+ synchronized(pendingUtterances) {
+ for (pair in pendingUtterances) {
+ speakInternal(pair.first, pair.second)
+ }
+ pendingUtterances.clear()
+ }
+ } else {
+ Log.e("NativeTtsManager", "Initialization Failed")
+ synchronized(pendingUtterances) {
+ pendingUtterances.clear()
+ }
+ onSpeakingStateChanged(false)
+ }
+ }
+ }
+
+ override fun speak(text: String, queueMode: Int) {
+ val tts = textToSpeech
+ if (tts == null) {
+ Log.w("NativeTtsManager", "TTS is null")
+ return
+ }
+
+ if (!isInitialized) {
+ Log.i("NativeTtsManager", "TTS not initialized yet. Queueing utterance: $text")
+ synchronized(pendingUtterances) {
+ if (queueMode == TtsManager.QUEUE_FLUSH) {
+ pendingUtterances.clear()
+ }
+ pendingUtterances.add(Pair(text, queueMode))
+ }
+ onSpeakingStateChanged(true)
+ return
+ }
+
+ speakInternal(text, queueMode)
+ }
+
+ private fun speakInternal(text: String, queueMode: Int) {
+ val tts = textToSpeech ?: return
+
+ val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
+ val useVoiceCall = audioManager.isBluetoothScoOn ||
+ audioManager.mode == AudioManager.MODE_IN_CALL ||
+ audioManager.mode == AudioManager.MODE_IN_COMMUNICATION
+
+ val audioAttributes = AudioAttributes.Builder()
+ .setUsage(if (useVoiceCall) AudioAttributes.USAGE_VOICE_COMMUNICATION else AudioAttributes.USAGE_MEDIA)
+ .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
+ .build()
+ tts.setAudioAttributes(audioAttributes)
+
+ val mode = if (queueMode == TtsManager.QUEUE_ADD) {
+ TextToSpeech.QUEUE_ADD
+ } else {
+ activeUtterances.clear()
+ TextToSpeech.QUEUE_FLUSH
+ }
+
+ val utteranceId = "tts_utterance_id_${System.currentTimeMillis()}"
+ activeUtterances.add(utteranceId)
+
+ val result = tts.speak(text, mode, null, utteranceId)
+ if (result != TextToSpeech.SUCCESS) {
+ activeUtterances.remove(utteranceId)
+ if (activeUtterances.isEmpty()) {
+ onSpeakingStateChanged(false)
+ }
+ }
+ }
+
+ override fun stop() {
+ synchronized(pendingUtterances) {
+ pendingUtterances.clear()
+ }
+ if (isInitialized) {
+ activeUtterances.clear()
+ val wasSpeaking = isSpeaking()
+ textToSpeech?.stop()
+ if (wasSpeaking) {
+ onSpeakingStateChanged(false)
+ }
+ } else {
+ onSpeakingStateChanged(false)
+ }
+ }
+
+ override fun isSpeaking(): Boolean {
+ return (isInitialized && (textToSpeech?.isSpeaking == true || activeUtterances.isNotEmpty())) ||
+ pendingUtterances.isNotEmpty()
+ }
+
+ override fun shutdown() {
+ synchronized(pendingUtterances) {
+ pendingUtterances.clear()
+ }
+ activeUtterances.clear()
+ textToSpeech?.let {
+ it.stop()
+ it.shutdown()
+ }
+ textToSpeech = null
+ isInitialized = false
+ }
+}
\ No newline at end of file
diff --git a/app/src/main/java/com/app/assistant/tts/OfflineTtsManager.kt b/app/src/main/java/com/app/assistant/tts/OfflineTtsManager.kt
new file mode 100644
index 0000000..7c86673
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/tts/OfflineTtsManager.kt
@@ -0,0 +1,397 @@
+package com.app.assistant.tts
+
+import android.content.Context
+import android.media.AudioAttributes
+import android.media.AudioFormat
+import android.media.AudioManager
+import android.media.AudioTrack
+import android.util.Log
+import com.app.assistant.repository.SettingsRepository
+import kotlinx.coroutines.*
+import kotlinx.coroutines.channels.Channel
+import java.io.File
+import com.k2fsa.sherpa.onnx.*
+import kotlin.coroutines.coroutineContext
+
+@OptIn(ExperimentalCoroutinesApi::class, ObsoleteCoroutinesApi::class, DelicateCoroutinesApi::class)
+class OfflineTtsManager(
+ private val context: Context,
+ private val settingsRepository: SettingsRepository,
+ private val onSpeakingStateChanged: (isSpeaking: Boolean) -> Unit
+) : TtsManager {
+
+ private val ttsDispatcher = newSingleThreadContext("OfflineTtsThread")
+ private val job = SupervisorJob()
+ private val scope = CoroutineScope(ttsDispatcher + job)
+
+ private val speechQueue = Channel(Channel.UNLIMITED)
+ private val audioQueue = Channel(Channel.UNLIMITED)
+
+ private var offlineTts: OfflineTts? = null
+ private var audioTrack: AudioTrack? = null
+
+ private var generatorJob: Job? = null
+ private var playbackJob: Job? = null
+
+ @Volatile
+ private var isSpeaking = false
+ @Volatile
+ private var isGenerating = false
+ @Volatile
+ private var isPlaying = false
+ private var isInitialized = false
+
+ private class GeneratedAudio(val samples: ShortArray, val sampleRate: Int)
+
+ init {
+ startQueueProcessing()
+ }
+
+ @Synchronized
+ private fun initTtsEngine(): Boolean {
+ if (isInitialized && offlineTts != null) return true
+
+ try {
+ val modelConfig: OfflineTtsModelConfig
+
+ val supertonicDir = File(context.filesDir, "sherpa-onnx-tts/sherpa-onnx-supertonic-3-tts-int8-2026-05-11")
+ val durationPredictor = File(supertonicDir, "duration_predictor.int8.onnx")
+ val textEncoder = File(supertonicDir, "text_encoder.int8.onnx")
+ val vectorEstimator = File(supertonicDir, "vector_estimator.int8.onnx")
+ val vocoder = File(supertonicDir, "vocoder.int8.onnx")
+ val ttsJson = File(supertonicDir, "tts.json")
+ val unicodeIndexer = File(supertonicDir, "unicode_indexer.bin")
+ val voiceStyle = File(supertonicDir, "voice.bin")
+
+ if (!durationPredictor.exists() || !textEncoder.exists() || !vectorEstimator.exists() ||
+ !vocoder.exists() || !ttsJson.exists() || !unicodeIndexer.exists() || !voiceStyle.exists()) {
+ Log.w("OfflineTtsManager", "Cannot init OfflineTts (SUPERTONIC): Missing files. durationPredictor: ${durationPredictor.exists()}, textEncoder: ${textEncoder.exists()}, vectorEstimator: ${vectorEstimator.exists()}, vocoder: ${vocoder.exists()}, ttsJson: ${ttsJson.exists()}, unicodeIndexer: ${unicodeIndexer.exists()}, voiceStyle: ${voiceStyle.exists()}")
+ return false
+ }
+
+ val supertonicConfig = OfflineTtsSupertonicModelConfig(
+ durationPredictor = durationPredictor.absolutePath,
+ textEncoder = textEncoder.absolutePath,
+ vectorEstimator = vectorEstimator.absolutePath,
+ vocoder = vocoder.absolutePath,
+ ttsJson = ttsJson.absolutePath,
+ unicodeIndexer = unicodeIndexer.absolutePath,
+ voiceStyle = voiceStyle.absolutePath
+ )
+ modelConfig = OfflineTtsModelConfig(
+ supertonic = supertonicConfig,
+ numThreads = 2,
+ debug = true
+ )
+
+ val config = OfflineTtsConfig(
+ model = modelConfig,
+ maxNumSentences = 1
+ )
+
+ // JNI initialization requires AssetManager to be null when loading absolute paths
+ offlineTts = OfflineTts(null, config)
+ isInitialized = true
+ Log.i("OfflineTtsManager", "Engine initialized successfully (type: SUPERTONIC)")
+ return true
+ } catch (e: Exception) {
+ Log.e("OfflineTtsManager", "Failed to initialize OfflineTts engine", e)
+ return false
+ }
+ }
+
+ private fun cleanTextForTts(text: String): String {
+ return text
+ .replace("—", " - ")
+ .replace("–", " - ")
+ .replace("“", "")
+ .replace("”", "")
+ .replace("\"", "")
+ .replace("‘", "'")
+ .replace("’", "'")
+ .trim()
+ }
+
+ private fun startQueueProcessing() {
+ generatorJob?.cancel()
+ playbackJob?.cancel()
+
+ while (!audioQueue.isEmpty) {
+ audioQueue.tryReceive()
+ }
+
+ generatorJob = scope.launch {
+ for (text in speechQueue) {
+ if (!initTtsEngine()) {
+ Log.w("OfflineTtsManager", "Engine not initialized. Skipping text: $text")
+ continue
+ }
+
+ val tts = offlineTts ?: continue
+ val cleanedText = cleanTextForTts(text)
+ if (cleanedText.isEmpty()) continue
+
+ isGenerating = true
+ updateSpeakingState()
+
+ try {
+ val audio = withContext(Dispatchers.Default) {
+ tts.generate(cleanedText, 0, 1.0f)
+ }
+ val floatSamples = audio.samples
+ val sampleRate = audio.sampleRate
+
+ if (floatSamples.isNotEmpty()) {
+ val shortSamples = ShortArray(floatSamples.size)
+ for (i in floatSamples.indices) {
+ val sample = floatSamples[i]
+ val clamped = Math.max(-1.0f, Math.min(1.0f, sample))
+ shortSamples[i] = (clamped * 32767.0f).toInt().toShort()
+ }
+ audioQueue.send(GeneratedAudio(shortSamples, sampleRate))
+ }
+ } catch (e: CancellationException) {
+ break
+ } catch (e: Exception) {
+ Log.e("OfflineTtsManager", "Error generating speech", e)
+ }
+
+ if (speechQueue.isEmpty) {
+ isGenerating = false
+ updateSpeakingState()
+ }
+ }
+ }
+
+ playbackJob = scope.launch {
+ var activeTrack: AudioTrack? = null
+ var currentSampleRate = 0
+
+ try {
+ while (coroutineContext.isActive) {
+ val audio = audioQueue.receive()
+ isPlaying = true
+ updateSpeakingState()
+
+ if (activeTrack == null || currentSampleRate != audio.sampleRate) {
+ activeTrack?.let {
+ try { it.stop(); it.release() } catch (e: Exception) {}
+ }
+ currentSampleRate = audio.sampleRate
+ val minBufferSize = AudioTrack.getMinBufferSize(
+ currentSampleRate,
+ AudioFormat.CHANNEL_OUT_MONO,
+ AudioFormat.ENCODING_PCM_16BIT
+ )
+ val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
+ val useVoiceCall = audioManager.isBluetoothScoOn ||
+ audioManager.mode == AudioManager.MODE_IN_CALL ||
+ audioManager.mode == AudioManager.MODE_IN_COMMUNICATION
+
+ activeTrack = AudioTrack.Builder()
+ .setAudioAttributes(
+ AudioAttributes.Builder()
+ .setUsage(if (useVoiceCall) AudioAttributes.USAGE_VOICE_COMMUNICATION else AudioAttributes.USAGE_MEDIA)
+ .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
+ .build()
+ )
+ .setAudioFormat(
+ AudioFormat.Builder()
+ .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
+ .setSampleRate(currentSampleRate)
+ .setChannelMask(AudioFormat.CHANNEL_OUT_MONO)
+ .build()
+ )
+ .setBufferSizeInBytes(Math.max(minBufferSize, audio.samples.size * 2))
+ .setTransferMode(AudioTrack.MODE_STREAM)
+ .build()
+ activeTrack.play()
+ audioTrack = activeTrack
+ }
+
+ // Write short samples to stream track
+ activeTrack.write(audio.samples, 0, audio.samples.size, AudioTrack.WRITE_BLOCKING)
+
+ // If there is no more audio in the queue, and we are not generating more,
+ // we wait for the current audio block to finish playing before releasing the track.
+ if (audioQueue.isEmpty && !isGenerating && speechQueue.isEmpty) {
+ val durationMs = (audio.samples.size.toDouble() / currentSampleRate * 1000).toLong()
+ delay(durationMs)
+
+ // Check again to ensure no new audio arrived during the delay
+ if (audioQueue.isEmpty && !isGenerating && speechQueue.isEmpty) {
+ try {
+ activeTrack.stop()
+ activeTrack.release()
+ } catch (e: Exception) {}
+ activeTrack = null
+ audioTrack = null
+ isPlaying = false
+ updateSpeakingState()
+ }
+ }
+ }
+ } catch (e: CancellationException) {
+ // normal cancellation
+ } catch (e: Exception) {
+ Log.e("OfflineTtsManager", "Error playing audio stream", e)
+ } finally {
+ activeTrack?.let {
+ try { it.stop(); it.release() } catch (e: Exception) {}
+ }
+ audioTrack = null
+ isPlaying = false
+ updateSpeakingState()
+ }
+ }
+ }
+
+ private fun updateSpeakingState() {
+ val shouldBeSpeaking = isGenerating || isPlaying || !speechQueue.isEmpty || !audioQueue.isEmpty
+ if (shouldBeSpeaking != isSpeaking) {
+ isSpeaking = shouldBeSpeaking
+ scope.launch(Dispatchers.Main) {
+ onSpeakingStateChanged(shouldBeSpeaking)
+ }
+ }
+ }
+
+ override fun speak(text: String, queueMode: Int) {
+ scope.launch {
+ if (queueMode == TtsManager.QUEUE_FLUSH) {
+ stopPlaybackOnly()
+ }
+ val sentences = splitIntoSentences(text)
+ for (sentence in sentences) {
+ val cleanedText = cleanTextForTts(sentence)
+ if (cleanedText.isNotEmpty()) {
+ speechQueue.send(cleanedText)
+ }
+ }
+ updateSpeakingState()
+ }
+ }
+
+ private fun splitIntoSentences(text: String): List {
+ val sentences = mutableListOf()
+ val currentSentence = StringBuilder()
+ val len = text.length
+ var i = 0
+ while (i < len) {
+ val c = text[i]
+ currentSentence.append(c)
+
+ // Split on newline or carriage return
+ if (c == '\n' || c == '\r') {
+ val sentenceStr = currentSentence.toString().trim()
+ if (sentenceStr.isNotEmpty()) {
+ sentences.add(sentenceStr)
+ }
+ currentSentence.clear()
+ i++
+ continue
+ }
+
+ // Split on sentence-ending punctuation followed by whitespace or end of text
+ if (c == '.' || c == '?' || c == '!' || c == ':' || c == ';') {
+ var isBoundary = false
+ if (i + 1 == len) {
+ isBoundary = true
+ } else {
+ val nextChar = text[i + 1]
+ if (nextChar.isWhitespace()) {
+ isBoundary = true
+ }
+ }
+
+ if (isBoundary) {
+ // Check if it's an abbreviation or decimal number (only for period)
+ if (c == '.' && isAbbreviationOrDecimal(text, i)) {
+ // Not a boundary
+ } else {
+ val sentenceStr = currentSentence.toString().trim()
+ if (sentenceStr.isNotEmpty()) {
+ sentences.add(sentenceStr)
+ }
+ currentSentence.clear()
+ }
+ }
+ }
+ i++
+ }
+
+ // Add any remaining text
+ val remaining = currentSentence.toString().trim()
+ if (remaining.isNotEmpty()) {
+ sentences.add(remaining)
+ }
+
+ return sentences
+ }
+
+ private fun isAbbreviationOrDecimal(text: String, dotIndex: Int): Boolean {
+ if (dotIndex > 0 && dotIndex + 1 < text.length) {
+ if (text[dotIndex - 1].isDigit() && text[dotIndex + 1].isDigit()) {
+ return true
+ }
+ }
+ var start = dotIndex - 1
+ while (start >= 0 && text[start].isLetter()) {
+ start--
+ }
+ val word = text.substring(start + 1, dotIndex).lowercase()
+ val abbreviations = setOf(
+ "mr", "mrs", "ms", "dr", "prof", "sr", "jr", "eg", "ie", "vs", "etc", "st", "co",
+ "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p",
+ "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"
+ )
+ return word in abbreviations
+ }
+
+ private fun stopPlaybackOnly() {
+ while (!speechQueue.isEmpty) {
+ speechQueue.tryReceive()
+ }
+
+ try {
+ audioTrack?.let {
+ if (it.playState == AudioTrack.PLAYSTATE_PLAYING) {
+ it.stop()
+ }
+ it.release()
+ }
+ } catch (e: Exception) {
+ Log.e("OfflineTtsManager", "Error stopping AudioTrack", e)
+ }
+ audioTrack = null
+
+ startQueueProcessing()
+
+ isGenerating = false
+ isPlaying = false
+ updateSpeakingState()
+ }
+
+ override fun stop() {
+ stopPlaybackOnly()
+ }
+
+ override fun isSpeaking(): Boolean {
+ return isSpeaking
+ }
+
+ override fun shutdown() {
+ job.cancel()
+ stopPlaybackOnly()
+ try {
+ runBlocking(ttsDispatcher) {
+ offlineTts?.release()
+ offlineTts = null
+ }
+ } catch (e: Exception) {
+ Log.e("OfflineTtsManager", "Error releasing offline TTS", e)
+ }
+ isInitialized = false
+ ttsDispatcher.close()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/tts/TtsEngineSelector.kt b/app/src/main/java/com/app/assistant/tts/TtsEngineSelector.kt
new file mode 100644
index 0000000..be119be
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/tts/TtsEngineSelector.kt
@@ -0,0 +1,99 @@
+package com.app.assistant.tts
+
+import android.content.Context
+import com.app.assistant.repository.SettingsRepository
+import java.io.File
+
+class TtsEngineSelector(
+ private val context: Context,
+ private val settingsRepository: SettingsRepository,
+ private val okHttpClient: okhttp3.OkHttpClient = com.app.assistant.viewmodel.MainViewModelFactory.okHttpClient,
+ private val onSpeakingStateChanged: (isSpeaking: Boolean) -> Unit
+) : TtsManager {
+
+ private var nativeTtsManager: NativeTtsManager? = null
+ private var offlineTtsManager: OfflineTtsManager? = null
+ private var apiTtsManager: ApiTtsManager? = null
+
+ private fun getNativeTtsManager(): NativeTtsManager {
+ return nativeTtsManager ?: NativeTtsManager(context, onSpeakingStateChanged).also { nativeTtsManager = it }
+ }
+
+ private fun getOfflineTtsManager(): OfflineTtsManager {
+ return offlineTtsManager ?: OfflineTtsManager(context, settingsRepository, onSpeakingStateChanged).also { offlineTtsManager = it }
+ }
+
+ private fun getApiTtsManager(): ApiTtsManager {
+ return apiTtsManager ?: ApiTtsManager(context, settingsRepository, okHttpClient, onSpeakingStateChanged).also { apiTtsManager = it }
+ }
+
+ override fun speak(text: String, queueMode: Int) {
+ val mode = settingsRepository.getTtsMode()
+
+ if (queueMode == TtsManager.QUEUE_FLUSH) {
+ nativeTtsManager?.stop()
+ offlineTtsManager?.stop()
+ apiTtsManager?.stop()
+ }
+
+ when (mode) {
+ TtsMode.OFFLINE -> {
+ if (isOfflineModelInstalled()) {
+ getOfflineTtsManager().speak(text, queueMode)
+ } else {
+ getNativeTtsManager().speak(text, queueMode)
+ }
+ }
+ TtsMode.API -> {
+ getApiTtsManager().speak(text, queueMode)
+ }
+ else -> {
+ getNativeTtsManager().speak(text, queueMode)
+ }
+ }
+ }
+
+ override fun stop() {
+ nativeTtsManager?.stop()
+ offlineTtsManager?.stop()
+ apiTtsManager?.stop()
+ }
+
+ override fun isSpeaking(): Boolean {
+ val mode = settingsRepository.getTtsMode()
+ return when (mode) {
+ TtsMode.OFFLINE -> {
+ if (isOfflineModelInstalled()) {
+ offlineTtsManager?.isSpeaking() ?: false
+ } else {
+ nativeTtsManager?.isSpeaking() ?: false
+ }
+ }
+ TtsMode.API -> {
+ apiTtsManager?.isSpeaking() ?: false
+ }
+ else -> {
+ nativeTtsManager?.isSpeaking() ?: false
+ }
+ }
+ }
+
+ override fun shutdown() {
+ nativeTtsManager?.shutdown()
+ offlineTtsManager?.shutdown()
+ apiTtsManager?.shutdown()
+ }
+
+ private fun isOfflineModelInstalled(): Boolean {
+ val supertonicDir = File(context.filesDir, "sherpa-onnx-tts/sherpa-onnx-supertonic-3-tts-int8-2026-05-11")
+ val durationPredictor = File(supertonicDir, "duration_predictor.int8.onnx")
+ val textEncoder = File(supertonicDir, "text_encoder.int8.onnx")
+ val vectorEstimator = File(supertonicDir, "vector_estimator.int8.onnx")
+ val vocoder = File(supertonicDir, "vocoder.int8.onnx")
+ val ttsJson = File(supertonicDir, "tts.json")
+ val unicodeIndexer = File(supertonicDir, "unicode_indexer.bin")
+ val voiceStyle = File(supertonicDir, "voice.bin")
+ return durationPredictor.exists() && textEncoder.exists() && vectorEstimator.exists() &&
+ vocoder.exists() && ttsJson.exists() && unicodeIndexer.exists() && voiceStyle.exists()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/tts/TtsManager.kt b/app/src/main/java/com/app/assistant/tts/TtsManager.kt
new file mode 100644
index 0000000..e513a1b
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/tts/TtsManager.kt
@@ -0,0 +1,19 @@
+package com.app.assistant.tts
+
+interface TtsManager {
+ fun speak(text: String, queueMode: Int = QUEUE_FLUSH)
+ fun stop()
+ fun isSpeaking(): Boolean
+ fun shutdown()
+
+ companion object {
+ const val QUEUE_FLUSH = 0
+ const val QUEUE_ADD = 1
+ }
+}
+
+enum class TtsMode {
+ NATIVE,
+ OFFLINE,
+ API
+}
diff --git a/app/src/main/java/com/app/assistant/tts/TtsModelManager.kt b/app/src/main/java/com/app/assistant/tts/TtsModelManager.kt
new file mode 100644
index 0000000..6f1e0df
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/tts/TtsModelManager.kt
@@ -0,0 +1,178 @@
+package com.app.assistant.tts
+
+import android.content.Context
+import android.os.Build
+import android.util.Log
+import okhttp3.OkHttpClient
+import okhttp3.Request
+import java.io.File
+import java.io.FileOutputStream
+import java.io.IOException
+import java.util.concurrent.atomic.AtomicBoolean
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+
+class TtsModelManager(private val context: Context) {
+
+ private val baseDir = File(context.filesDir, "sherpa-onnx-tts")
+ private val modelDir = File(baseDir, "sherpa-onnx-supertonic-3-tts-int8-2026-05-11")
+
+ private val downloadUrl = "https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/sherpa-onnx-supertonic-3-tts-int8-2026-05-11.tar.bz2"
+
+ private var currentCall: okhttp3.Call? = null
+ private val isCancelled = AtomicBoolean(false)
+
+ fun isModelDownloaded(): Boolean {
+ val durationPredictor = File(modelDir, "duration_predictor.int8.onnx")
+ val textEncoder = File(modelDir, "text_encoder.int8.onnx")
+ val vectorEstimator = File(modelDir, "vector_estimator.int8.onnx")
+ val vocoder = File(modelDir, "vocoder.int8.onnx")
+ val ttsJson = File(modelDir, "tts.json")
+ val unicodeIndexer = File(modelDir, "unicode_indexer.bin")
+ val voiceStyle = File(modelDir, "voice.bin")
+ return durationPredictor.exists() && durationPredictor.length() > 0 &&
+ textEncoder.exists() && textEncoder.length() > 0 &&
+ vectorEstimator.exists() && vectorEstimator.length() > 0 &&
+ vocoder.exists() && vocoder.length() > 0 &&
+ ttsJson.exists() && ttsJson.length() > 0 &&
+ unicodeIndexer.exists() && unicodeIndexer.length() > 0 &&
+ voiceStyle.exists() && voiceStyle.length() > 0
+ }
+
+ fun deleteLocalModel(): Boolean {
+ return deleteDirectory(modelDir)
+ }
+
+ private fun deleteDirectory(dir: File): Boolean {
+ if (dir.isDirectory) {
+ val children = dir.listFiles()
+ if (children != null) {
+ for (child in children) {
+ deleteDirectory(child)
+ }
+ }
+ }
+ return dir.delete()
+ }
+
+ suspend fun downloadModel(
+ client: OkHttpClient,
+ onProgress: (downloadedBytes: Long, totalBytes: Long) -> Unit,
+ onComplete: (success: Boolean, errorMessage: String?) -> Unit
+ ) {
+ isCancelled.set(false)
+ withContext(Dispatchers.IO) {
+ val tempArchive = File(context.cacheDir, "sherpa-onnx-supertonic-3-tts-int8-2026-05-11.tar.bz2")
+ try {
+ if (tempArchive.exists()) {
+ tempArchive.delete()
+ }
+
+ if (!baseDir.exists()) {
+ baseDir.mkdirs()
+ }
+
+ val request = Request.Builder()
+ .url(downloadUrl)
+ .build()
+
+ val call = client.newCall(request)
+ currentCall = call
+
+ if (isCancelled.get()) {
+ throw IOException("Download cancelled by user")
+ }
+
+ val response = call.execute()
+ if (!response.isSuccessful) {
+ throw IOException("HTTP error code: ${response.code}")
+ }
+
+ val responseBody = response.body ?: throw IOException("Empty response body")
+ val totalBytes = responseBody.contentLength()
+
+ responseBody.byteStream().use { inputStream ->
+ FileOutputStream(tempArchive).use { outputStream ->
+ val buffer = ByteArray(1024 * 64)
+ var bytesRead: Int
+ var downloadedBytes = 0L
+
+ while (inputStream.read(buffer).also { bytesRead = it } != -1) {
+ if (isCancelled.get()) {
+ throw IOException("Download cancelled by user")
+ }
+ outputStream.write(buffer, 0, bytesRead)
+ downloadedBytes += bytesRead
+ onProgress(downloadedBytes, totalBytes)
+ }
+ outputStream.flush()
+ }
+ }
+ response.close()
+
+ if (isCancelled.get()) {
+ throw IOException("Download cancelled by user")
+ }
+
+ // Extract archive
+ onProgress(totalBytes, totalBytes) // Mark download as 100% before extraction
+ val extractionSuccess = extractTarBz2(tempArchive, baseDir)
+
+ if (extractionSuccess && isModelDownloaded()) {
+ onComplete(true, null)
+ } else {
+ onComplete(false, "Extraction failed. Ensure your device has enough free storage.")
+ }
+ } catch (e: Exception) {
+ Log.e("TtsModelManager", "Error downloading/extracting TTS model", e)
+ val msg = if (isCancelled.get()) "Download cancelled." else e.message ?: "Unknown error"
+ onComplete(false, msg)
+ } finally {
+ currentCall = null
+ if (tempArchive.exists()) {
+ tempArchive.delete()
+ }
+ }
+ }
+ }
+
+ fun cancelDownload() {
+ isCancelled.set(true)
+ currentCall?.cancel()
+ currentCall = null
+ }
+
+ private fun extractTarBz2(archiveFile: File, targetDir: File): Boolean {
+ targetDir.mkdirs()
+ try {
+ Log.i("TtsModelManager", "Starting native tar extraction of ${archiveFile.name} to ${targetDir.name}...")
+ val process = ProcessBuilder()
+ .command("tar", "-xjf", archiveFile.absolutePath, "-C", targetDir.absolutePath)
+ .redirectErrorStream(true)
+ .start()
+
+ val exitCode = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
+ if (process.waitFor(90, java.util.concurrent.TimeUnit.SECONDS)) { // 90 secs timeout for slower devices
+ process.exitValue()
+ } else {
+ process.destroyForcibly()
+ -1
+ }
+ } else {
+ process.waitFor()
+ }
+
+ if (exitCode == 0) {
+ Log.i("TtsModelManager", "Native tar extraction completed successfully.")
+ return true
+ }
+
+ val reader = process.inputStream.bufferedReader()
+ val output = reader.readText()
+ Log.e("TtsModelManager", "Native tar failed with exit code $exitCode. Output: $output")
+ } catch (e: Exception) {
+ Log.e("TtsModelManager", "Failed to run native tar command", e)
+ }
+ return false
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/screen/CameraPreviewContainer.kt b/app/src/main/java/com/app/assistant/ui/screen/CameraPreviewContainer.kt
new file mode 100644
index 0000000..b0f0f6c
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/CameraPreviewContainer.kt
@@ -0,0 +1,225 @@
+package com.app.assistant.ui.screen
+
+import android.util.Log
+import android.util.Size
+import androidx.camera.core.CameraSelector
+import androidx.camera.core.FocusMeteringAction
+import androidx.camera.core.ImageAnalysis
+import androidx.camera.core.Preview
+import androidx.camera.core.resolutionselector.ResolutionSelector
+import androidx.camera.core.resolutionselector.ResolutionStrategy
+import androidx.camera.lifecycle.ProcessCameraProvider
+import androidx.camera.view.PreviewView
+import androidx.compose.animation.core.Animatable
+import androidx.compose.animation.core.tween
+import androidx.compose.foundation.background
+import androidx.compose.foundation.border
+import androidx.compose.foundation.gestures.detectTapGestures
+import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.fillMaxSize
+import androidx.compose.foundation.layout.size
+import androidx.compose.foundation.layout.padding
+import androidx.compose.material3.MaterialTheme
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.DisposableEffect
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.mutableStateOf
+import androidx.compose.runtime.remember
+import androidx.compose.runtime.rememberCoroutineScope
+import androidx.compose.runtime.setValue
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.geometry.Offset
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.graphics.graphicsLayer
+import androidx.compose.ui.input.pointer.pointerInput
+import androidx.compose.ui.platform.LocalContext
+import androidx.compose.ui.platform.LocalLifecycleOwner
+import androidx.compose.ui.unit.dp
+import androidx.compose.ui.viewinterop.AndroidView
+import androidx.core.content.ContextCompat
+import com.app.assistant.camera.VisualBufferManager
+import com.app.assistant.camera.imageProxyToRotatedJpeg
+import kotlinx.coroutines.launch
+import java.util.concurrent.Executors
+import java.util.concurrent.TimeUnit
+
+
+@Composable
+fun CameraPreviewContainer(
+ modifier: Modifier = Modifier,
+ lensFacing: Int = CameraSelector.LENS_FACING_BACK
+) {
+ val context = LocalContext.current
+ val lifecycleOwner = LocalLifecycleOwner.current
+ val coroutineScope = rememberCoroutineScope()
+
+ var activeCamera by remember { mutableStateOf(null) }
+ var tapPoint by remember { mutableStateOf(null) }
+
+ val focusRingAlpha = remember { Animatable(0f) }
+ val focusRingScale = remember { Animatable(1f) }
+
+ val previewView = remember {
+ PreviewView(context).apply {
+ scaleType = PreviewView.ScaleType.FILL_CENTER
+ }
+ }
+
+ DisposableEffect(lifecycleOwner, lensFacing) {
+ var cameraExecutor: java.util.concurrent.ExecutorService? = null
+ val cameraProviderFuture = ProcessCameraProvider.getInstance(context)
+
+ val observer = androidx.lifecycle.LifecycleEventObserver { _, event ->
+ when (event) {
+ androidx.lifecycle.Lifecycle.Event.ON_RESUME -> {
+ val executor = Executors.newSingleThreadExecutor()
+ cameraExecutor = executor
+
+ cameraProviderFuture.addListener({
+ try {
+ val cameraProvider = cameraProviderFuture.get()
+
+ val preview = Preview.Builder().build().apply {
+ setSurfaceProvider(previewView.surfaceProvider)
+ }
+
+ val resolutionSelector = ResolutionSelector.Builder()
+ .setResolutionStrategy(
+ ResolutionStrategy(
+ Size(640, 480),
+ ResolutionStrategy.FALLBACK_RULE_CLOSEST_HIGHER_THEN_LOWER
+ )
+ )
+ .build()
+
+ val imageAnalysis = ImageAnalysis.Builder()
+ .setResolutionSelector(resolutionSelector)
+ .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
+ .build()
+
+ var lastFrameTimestamp = 0L
+
+ imageAnalysis.setAnalyzer(executor) { imageProxy ->
+ val now = System.currentTimeMillis()
+ if (now - lastFrameTimestamp < 150) {
+ imageProxy.close()
+ return@setAnalyzer
+ }
+ lastFrameTimestamp = now
+
+ val jpegBytes = imageProxyToRotatedJpeg(imageProxy)
+ imageProxy.close()
+
+ if (jpegBytes != null) {
+ val cachedFrame = com.app.assistant.camera.CachedFrame(jpegBytes, now)
+ VisualBufferManager.addFrame(cachedFrame)
+ }
+ }
+
+ val cameraSelector = if (lensFacing == CameraSelector.LENS_FACING_BACK) {
+ CameraSelector.DEFAULT_BACK_CAMERA
+ } else {
+ CameraSelector.DEFAULT_FRONT_CAMERA
+ }
+
+ cameraProvider.unbindAll()
+ val camera = cameraProvider.bindToLifecycle(
+ lifecycleOwner,
+ cameraSelector,
+ preview,
+ imageAnalysis
+ )
+ activeCamera = camera
+ } catch (e: Exception) {
+ Log.e("CameraPreview", "Use case binding failed", e)
+ }
+ }, ContextCompat.getMainExecutor(context))
+ }
+ androidx.lifecycle.Lifecycle.Event.ON_PAUSE -> {
+ try {
+ if (cameraProviderFuture.isDone) {
+ val cameraProvider = cameraProviderFuture.get()
+ cameraProvider.unbindAll()
+ }
+ } catch (e: Exception) {
+ Log.e("CameraPreview", "Error unbinding on ON_PAUSE", e)
+ }
+ cameraExecutor?.shutdown()
+ cameraExecutor = null
+ activeCamera = null
+ VisualBufferManager.clear()
+ }
+ else -> {}
+ }
+ }
+
+ lifecycleOwner.lifecycle.addObserver(observer)
+
+ onDispose {
+ lifecycleOwner.lifecycle.removeObserver(observer)
+ try {
+ if (cameraProviderFuture.isDone) {
+ val cameraProvider = cameraProviderFuture.get()
+ cameraProvider.unbindAll()
+ }
+ } catch (e: Exception) {
+ Log.e("CameraPreview", "Error unbinding on dispose", e)
+ }
+ cameraExecutor?.shutdown()
+ activeCamera = null
+ VisualBufferManager.clear()
+ }
+ }
+
+ Box(modifier = modifier) {
+ AndroidView(
+ factory = { previewView },
+ modifier = Modifier
+ .fillMaxSize()
+ .pointerInput(activeCamera) {
+ detectTapGestures { offset ->
+ val camera = activeCamera ?: return@detectTapGestures
+ tapPoint = offset
+
+ coroutineScope.launch {
+ focusRingAlpha.snapTo(1f)
+ focusRingScale.snapTo(1.5f)
+
+ focusRingScale.animateTo(1f, tween(300))
+ focusRingAlpha.animateTo(0f, tween(400, delayMillis = 100))
+ }
+
+ val factory = previewView.meteringPointFactory
+ val point = factory.createPoint(offset.x, offset.y)
+ val action = FocusMeteringAction.Builder(point, FocusMeteringAction.FLAG_AF or FocusMeteringAction.FLAG_AE)
+ .setAutoCancelDuration(5, TimeUnit.SECONDS)
+ .build()
+ camera.cameraControl.startFocusAndMetering(action)
+ }
+ }
+ )
+
+ tapPoint?.let { point ->
+ val scale = focusRingScale.value
+ val alpha = focusRingAlpha.value
+ if (alpha > 0f) {
+ Box(
+ modifier = Modifier
+ .size(64.dp)
+ .graphicsLayer {
+ translationX = point.x - size.width / 2f
+ translationY = point.y - size.height / 2f
+ scaleX = scale
+ scaleY = scale
+ this.alpha = alpha
+ }
+ .border(
+ width = 2.dp,
+ color = MaterialTheme.colorScheme.primary,
+ shape = androidx.compose.foundation.shape.CircleShape
+ )
+ )
+ }
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/screen/ChatLayout.kt b/app/src/main/java/com/app/assistant/ui/screen/ChatLayout.kt
new file mode 100644
index 0000000..a5118c3
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/ChatLayout.kt
@@ -0,0 +1,446 @@
+package com.app.assistant.ui.screen
+
+import android.os.Build
+import android.os.VibrationEffect
+import android.os.Vibrator
+import android.os.VibratorManager
+import android.widget.Toast
+import androidx.compose.foundation.layout.Arrangement
+import androidx.compose.foundation.layout.Column
+import androidx.compose.foundation.layout.fillMaxSize
+import androidx.compose.foundation.layout.height
+import androidx.compose.foundation.layout.padding
+import androidx.compose.foundation.lazy.LazyColumn
+import androidx.compose.foundation.lazy.itemsIndexed
+import androidx.compose.foundation.lazy.rememberLazyListState
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.LaunchedEffect
+import kotlinx.coroutines.delay
+import androidx.compose.ui.Alignment
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.platform.LocalContext
+import androidx.compose.ui.platform.LocalFocusManager
+import androidx.compose.ui.platform.LocalSoftwareKeyboardController
+import androidx.compose.ui.unit.dp
+import com.app.assistant.model.Conversation
+import com.app.assistant.R
+import androidx.camera.core.CameraSelector
+import androidx.compose.ui.tooling.preview.Preview
+import com.app.assistant.ui.theme.AssistantTheme
+import androidx.compose.ui.focus.FocusManager
+import androidx.compose.ui.platform.SoftwareKeyboardController
+import androidx.compose.foundation.background
+import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.Row
+import androidx.compose.foundation.layout.fillMaxWidth
+import androidx.compose.foundation.layout.size
+import androidx.compose.foundation.shape.CircleShape
+import androidx.compose.foundation.shape.RoundedCornerShape
+import androidx.compose.material.icons.Icons
+import androidx.compose.material.icons.filled.Warning
+import androidx.compose.material.icons.filled.Close
+import androidx.compose.ui.res.painterResource
+import androidx.compose.material3.Icon
+import androidx.compose.material3.Text
+import androidx.compose.material3.MaterialTheme
+import androidx.compose.material3.IconButton
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.draw.clip
+import androidx.compose.runtime.remember
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.setValue
+import androidx.compose.runtime.mutableStateOf
+import androidx.compose.animation.AnimatedContent
+import androidx.compose.animation.slideInVertically
+import androidx.compose.animation.slideOutVertically
+import androidx.compose.animation.fadeIn
+import androidx.compose.animation.fadeOut
+import androidx.compose.animation.togetherWith
+import androidx.compose.animation.core.tween
+
+@Composable
+@Suppress("ktlint:standard:function-naming")
+fun ChatLayout(
+ modifier: Modifier = Modifier,
+ onShowCopyIconChange: (Boolean) -> Unit,
+ selectedItemIndex: Int?,
+ onSelectedItemChange: (Int) -> Unit,
+ isCustomUI: Boolean,
+ chatList: List,
+ isSpeaking: Boolean,
+ isListening: Boolean,
+ audioAmplitude: Float = 0f,
+ question: String,
+ onQuestionChange: (String) -> Unit,
+ onStopSpeaking: () -> Unit,
+ onStartListening: () -> Unit,
+ onProcessQuestion: (FocusManager, SoftwareKeyboardController, Boolean) -> Unit,
+ isTranslateEnabled: Boolean,
+ selectedAttachments: List = emptyList(),
+ onRemoveAttachment: (com.app.assistant.model.Attachment) -> Unit = {},
+ onAttachClick: (String) -> Unit = {},
+ isImageSupported: Boolean = false,
+ isAudioSupported: Boolean = false,
+ isVideoSupported: Boolean = false,
+ isDocumentSupported: Boolean = false,
+ isHandsFree: Boolean = false,
+ onToggleHandsFree: () -> Unit = {},
+ onExitHandsFree: () -> Unit = {},
+ isVoiceProcessing: Boolean = false,
+ isMicReady: Boolean = false,
+ isMicMuted: Boolean = false,
+ onToggleMicMute: () -> Unit = {},
+ isVisionModeActive: Boolean = false,
+ onToggleVisionMode: () -> Unit = {},
+ isScreenModeActive: Boolean = false,
+ onToggleScreenMode: () -> Unit = {},
+) {
+ val keyboardController = LocalSoftwareKeyboardController.current
+ val focusManager = LocalFocusManager.current
+ val scrollState = rememberLazyListState()
+
+ var isCameraExpanded by remember(isVisionModeActive) { mutableStateOf(true) }
+ var lensFacing by remember { mutableStateOf(CameraSelector.LENS_FACING_BACK) }
+
+ val context = LocalContext.current
+ val vibrator =
+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
+ context.getSystemService(VibratorManager::class.java)?.defaultVibrator
+ } else {
+ @Suppress("DEPRECATION")
+ context.getSystemService(Vibrator::class.java)
+ }
+
+ LaunchedEffect(selectedItemIndex) {
+ onShowCopyIconChange(selectedItemIndex != null)
+ if (selectedItemIndex != null) {
+ if (vibrator?.hasVibrator() == true) {
+ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
+ vibrator.vibrate(VibrationEffect.createOneShot(50, VibrationEffect.DEFAULT_AMPLITUDE))
+ } else {
+ @Suppress("DEPRECATION")
+ vibrator.vibrate(50)
+ }
+ }
+ }
+ }
+
+ Box(modifier = modifier.fillMaxSize()) {
+ if (isVisionModeActive && isCameraExpanded) {
+ CameraPreviewContainer(
+ modifier = Modifier.fillMaxSize(),
+ lensFacing = lensFacing
+ )
+ }
+
+ Column(modifier = Modifier.fillMaxSize()) {
+ Box(
+ modifier = Modifier
+ .weight(1f)
+ .fillMaxWidth()
+ ) {
+ if (!(isVisionModeActive && isCameraExpanded)) {
+ Column(modifier = Modifier.fillMaxSize()) {
+ LazyColumn(
+ modifier =
+ Modifier
+ .weight(1f)
+ .padding(0.dp, 0.dp, 0.dp, 0.dp),
+ verticalArrangement =
+ if (isCustomUI) {
+ Arrangement.spacedBy(8.dp, Alignment.Bottom)
+ } else {
+ Arrangement.spacedBy(8.dp)
+ },
+ state = scrollState,
+ ) {
+ itemsIndexed(chatList) { index, conversation ->
+ ConversationItem(
+ conversation = conversation,
+ index = index,
+ isSelected = index == selectedItemIndex,
+ onLongClick = { newIndex -> onSelectedItemChange(newIndex) },
+ isTranslateEnabled = isTranslateEnabled,
+ )
+ }
+ }
+
+ var isDismissed by remember(chatList.size, isImageSupported, isAudioSupported, isVideoSupported, isDocumentSupported) {
+ mutableStateOf(false)
+ }
+
+ val showWarning = remember(chatList.size, isImageSupported, isAudioSupported, isVideoSupported, isDocumentSupported) {
+ chatList.any { conversation ->
+ conversation.attachments.any { att ->
+ val isTextDoc = att.mimeType.startsWith("text/") || att.mimeType == "application/json"
+ if (isTextDoc) {
+ false
+ } else {
+ when {
+ att.mimeType.startsWith("image/") -> !isImageSupported
+ att.mimeType.startsWith("audio/") -> !isAudioSupported
+ att.mimeType.startsWith("video/") -> !isVideoSupported
+ att.mimeType.startsWith("application/pdf") -> !isDocumentSupported
+ else -> true
+ }
+ }
+ }
+ }
+ }
+
+ if (showWarning && !isDismissed) {
+ Row(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(horizontal = 16.dp, vertical = 4.dp)
+ .background(
+ color = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.6f),
+ shape = RoundedCornerShape(12.dp)
+ )
+ .padding(12.dp),
+ verticalAlignment = Alignment.CenterVertically,
+ horizontalArrangement = Arrangement.spacedBy(8.dp)
+ ) {
+ Icon(
+ imageVector = Icons.Default.Warning,
+ contentDescription = "Warning",
+ tint = MaterialTheme.colorScheme.primary,
+ modifier = Modifier.size(20.dp)
+ )
+ Text(
+ text = "Current model does not support some media in this chat. Incompatible past attachments will be ignored by the LLM.",
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant,
+ modifier = Modifier.weight(1f)
+ )
+ IconButton(
+ onClick = { isDismissed = true },
+ modifier = Modifier.size(20.dp)
+ ) {
+ Icon(
+ imageVector = Icons.Default.Close,
+ contentDescription = "Dismiss warning",
+ tint = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.7f),
+ modifier = Modifier.size(16.dp)
+ )
+ }
+ }
+ }
+
+ if (isVisionModeActive) {
+ Box(
+ modifier = Modifier
+ .fillMaxWidth()
+ .height(200.dp)
+ .padding(horizontal = 16.dp, vertical = 8.dp)
+ .clip(RoundedCornerShape(16.dp))
+ .background(Color.Black)
+ ) {
+ CameraPreviewContainer(
+ modifier = Modifier.fillMaxSize(),
+ lensFacing = lensFacing
+ )
+ Column(
+ modifier = Modifier
+ .align(Alignment.BottomEnd)
+ .padding(8.dp),
+ verticalArrangement = Arrangement.spacedBy(8.dp),
+ horizontalAlignment = Alignment.CenterHorizontally
+ ) {
+ IconButton(
+ onClick = {
+ lensFacing = if (lensFacing == CameraSelector.LENS_FACING_BACK) {
+ CameraSelector.LENS_FACING_FRONT
+ } else {
+ CameraSelector.LENS_FACING_BACK
+ }
+ },
+ modifier = Modifier
+ .background(Color.Black.copy(alpha = 0.5f), CircleShape)
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_switch_camera),
+ contentDescription = "Switch Camera",
+ tint = Color.White
+ )
+ }
+ IconButton(
+ onClick = { isCameraExpanded = true },
+ modifier = Modifier
+ .background(Color.Black.copy(alpha = 0.5f), CircleShape)
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_fullscreen),
+ contentDescription = "Expand Camera Preview",
+ tint = Color.White
+ )
+ }
+ }
+ }
+ }
+ }
+ } else {
+ // Expanded mode overlay buttons stacked on bottom right
+ Column(
+ modifier = Modifier
+ .align(Alignment.BottomEnd)
+ .padding(16.dp),
+ verticalArrangement = Arrangement.spacedBy(8.dp),
+ horizontalAlignment = Alignment.CenterHorizontally
+ ) {
+ IconButton(
+ onClick = {
+ lensFacing = if (lensFacing == CameraSelector.LENS_FACING_BACK) {
+ CameraSelector.LENS_FACING_FRONT
+ } else {
+ CameraSelector.LENS_FACING_BACK
+ }
+ },
+ modifier = Modifier
+ .background(Color.Black.copy(alpha = 0.5f), CircleShape)
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_switch_camera),
+ contentDescription = "Switch Camera",
+ tint = Color.White
+ )
+ }
+ IconButton(
+ onClick = { isCameraExpanded = false },
+ modifier = Modifier
+ .background(Color.Black.copy(alpha = 0.5f), CircleShape)
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_fullscreen_exit),
+ contentDescription = "Collapse Camera Preview",
+ tint = Color.White
+ )
+ }
+ }
+ }
+ }
+
+ AnimatedContent(
+ targetState = isHandsFree,
+ transitionSpec = {
+ (slideInVertically(animationSpec = tween(300)) { height -> height } + fadeIn(animationSpec = tween(300)))
+ .togetherWith(
+ slideOutVertically(animationSpec = tween(300)) { height -> height } + fadeOut(animationSpec = tween(300))
+ )
+ },
+ label = "InputFieldTransition",
+ modifier = Modifier.fillMaxWidth()
+ ) { handsFreeActive ->
+ if (handsFreeActive) {
+ HandsFreeBar(
+ isSpeaking = isSpeaking,
+ isListening = isListening,
+ audioAmplitude = audioAmplitude,
+ isThinking = isVoiceProcessing || chatList.any { it.isLoading || it.isStreaming },
+ isMicReady = isMicReady,
+ onExitHandsFree = onExitHandsFree,
+ isMuted = isMicMuted,
+ onToggleMute = onToggleMicMute,
+ isVisionModeActive = isVisionModeActive,
+ onToggleVisionMode = onToggleVisionMode,
+ isScreenModeActive = isScreenModeActive,
+ onToggleScreenMode = onToggleScreenMode,
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(8.dp)
+ )
+ } else {
+ if (keyboardController != null) {
+ UserInputField(
+ focusManager = focusManager,
+ keyboardController = keyboardController,
+ isSpeaking = isSpeaking,
+ isListening = isListening,
+ question = question,
+ onQuestionChange = onQuestionChange,
+ onStopSpeaking = onStopSpeaking,
+ onStartListening = onStartListening,
+ onProcessQuestion = onProcessQuestion,
+ selectedAttachments = selectedAttachments,
+ onRemoveAttachment = onRemoveAttachment,
+ onAttachClick = onAttachClick,
+ isImageSupported = isImageSupported,
+ isAudioSupported = isAudioSupported,
+ isVideoSupported = isVideoSupported,
+ isDocumentSupported = isDocumentSupported,
+ isHandsFree = isHandsFree,
+ onToggleHandsFree = onToggleHandsFree,
+ hasMessages = chatList.isNotEmpty(),
+ )
+ }
+ }
+ }
+ }
+ }
+
+ val lastItem = chatList.lastOrNull()
+ LaunchedEffect(chatList.size, lastItem?.text, lastItem?.isStreaming, lastItem?.isLoading) {
+ if (chatList.isNotEmpty()) {
+ val lastIndex = chatList.size - 1
+
+ var lastVisibleItem = scrollState.layoutInfo.visibleItemsInfo.firstOrNull { it.index == lastIndex }
+ if (lastVisibleItem == null) {
+ if (lastItem?.isStreaming == true) {
+ scrollState.scrollToItem(lastIndex, 0)
+ } else {
+ scrollState.animateScrollToItem(lastIndex, 0)
+ }
+ delay(50)
+ lastVisibleItem = scrollState.layoutInfo.visibleItemsInfo.firstOrNull { it.index == lastIndex }
+ }
+
+ val itemHeight = lastVisibleItem?.size ?: 0
+ val viewportHeight = scrollState.layoutInfo.viewportEndOffset - scrollState.layoutInfo.viewportStartOffset
+ if (viewportHeight > 0 && itemHeight > 0) {
+ val scrollOffset = viewportHeight - itemHeight
+ if (lastItem?.isStreaming == true) {
+ scrollState.scrollToItem(lastIndex, scrollOffset)
+ } else {
+ scrollState.animateScrollToItem(lastIndex, scrollOffset)
+ }
+ } else {
+ if (lastItem?.isStreaming == true) {
+ scrollState.scrollToItem(lastIndex, 0)
+ } else {
+ scrollState.animateScrollToItem(lastIndex, 0)
+ }
+ }
+ }
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun ChatLayoutPreview() {
+ AssistantTheme {
+ ChatLayout(
+ onShowCopyIconChange = {},
+ selectedItemIndex = null,
+ onSelectedItemChange = {},
+ isCustomUI = false,
+ chatList = listOf(
+ Conversation(
+ text = "What is the weather today?",
+ isMe = true
+ ),
+ Conversation(
+ text = "The weather is sunny at 22°C.",
+ isMe = false
+ )
+ ),
+ isSpeaking = false,
+ isListening = false,
+ question = "",
+ onQuestionChange = {},
+ onStopSpeaking = {},
+ onStartListening = {},
+ onProcessQuestion = { _, _, _ -> },
+ isTranslateEnabled = false
+ )
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/screen/ChatScreen.kt b/app/src/main/java/com/app/assistant/ui/screen/ChatScreen.kt
index bc02392..014fcca 100644
--- a/app/src/main/java/com/app/assistant/ui/screen/ChatScreen.kt
+++ b/app/src/main/java/com/app/assistant/ui/screen/ChatScreen.kt
@@ -1,60 +1,31 @@
package com.app.assistant.ui.screen
-import android.os.Build
-import android.os.VibrationEffect
-import android.os.Vibrator
-import android.os.VibratorManager
-import android.util.Log
-import android.widget.Toast
import androidx.activity.compose.BackHandler
import androidx.activity.compose.LocalOnBackPressedDispatcherOwner
import androidx.compose.animation.animateContentSize
-import androidx.compose.animation.core.Animatable
-import androidx.compose.animation.core.FastOutSlowInEasing
-import androidx.compose.animation.core.LinearEasing
-import androidx.compose.animation.core.RepeatMode
-import androidx.compose.animation.core.StartOffset
-import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.animateFloatAsState
-import androidx.compose.animation.core.infiniteRepeatable
-import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
-import androidx.compose.foundation.Canvas
import androidx.compose.foundation.background
-import androidx.compose.foundation.border
import androidx.compose.foundation.clickable
import androidx.compose.foundation.gestures.detectDragGestures
-import androidx.compose.foundation.gestures.detectTapGestures
-import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
-import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.imePadding
-import androidx.compose.foundation.layout.offset
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.width
-import androidx.compose.foundation.layout.widthIn
-import androidx.compose.foundation.layout.wrapContentWidth
+import androidx.compose.foundation.layout.Arrangement
+import androidx.compose.foundation.layout.Spacer
+import androidx.compose.foundation.layout.size
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
-import androidx.compose.foundation.lazy.itemsIndexed
-import androidx.compose.foundation.lazy.rememberLazyListState
import androidx.compose.foundation.rememberScrollState
-import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.verticalScroll
-import androidx.compose.material.icons.Icons
-import androidx.compose.material.icons.automirrored.filled.Send
-import androidx.compose.material.icons.filled.Call
-import androidx.compose.material.icons.filled.LocationOn
import androidx.compose.material3.AlertDialog
-import androidx.compose.material3.Button
-import androidx.compose.material3.Card
-import androidx.compose.material3.CardDefaults
import androidx.compose.material3.DrawerState
import androidx.compose.material3.DrawerValue
import androidx.compose.material3.ExperimentalMaterial3Api
@@ -74,8 +45,22 @@ import androidx.compose.material3.TextButton
import androidx.compose.material3.TextField
import androidx.compose.material3.TextFieldDefaults
import androidx.compose.material3.TopAppBar
-import androidx.compose.material3.rememberDrawerState
import androidx.compose.material3.rememberModalBottomSheetState
+import androidx.compose.animation.core.InfiniteTransition
+import androidx.compose.animation.core.rememberInfiniteTransition
+import androidx.compose.animation.core.animateFloat
+import androidx.compose.animation.core.infiniteRepeatable
+import androidx.compose.animation.core.RepeatMode
+import androidx.compose.animation.core.FastOutSlowInEasing
+import androidx.compose.foundation.shape.CircleShape
+import androidx.compose.ui.text.style.TextAlign
+import androidx.compose.ui.text.style.TextOverflow
+import androidx.compose.foundation.BorderStroke
+import androidx.compose.material3.Button
+import androidx.compose.material3.ButtonDefaults
+import androidx.compose.material.icons.Icons
+import androidx.compose.material.icons.filled.Close
+import androidx.compose.ui.text.font.FontWeight
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
@@ -83,158 +68,293 @@ import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
-import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
-import androidx.compose.ui.draw.drawBehind
-import androidx.compose.ui.focus.FocusManager
-import androidx.compose.ui.geometry.CornerRadius
import androidx.compose.ui.graphics.Color
-import androidx.compose.ui.graphics.PaintingStyle.Companion.Stroke
-import androidx.compose.ui.graphics.drawscope.Stroke
-import androidx.compose.ui.graphics.luminance
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.layout.onGloballyPositioned
import androidx.compose.ui.platform.LocalClipboardManager
import androidx.compose.ui.platform.LocalConfiguration
import androidx.compose.ui.platform.LocalContext
-import androidx.compose.ui.platform.LocalFocusManager
-import androidx.compose.ui.platform.LocalSoftwareKeyboardController
-import androidx.compose.ui.platform.LocalUriHandler
-import androidx.compose.ui.platform.SoftwareKeyboardController
-import androidx.compose.ui.platform.UriHandler
import androidx.compose.ui.res.painterResource
+import androidx.compose.ui.res.stringResource
import androidx.compose.ui.text.AnnotatedString
-import androidx.compose.ui.text.SpanStyle
-import androidx.compose.ui.text.font.FontFamily
-import androidx.compose.ui.text.font.FontStyle
-import androidx.compose.ui.text.font.FontWeight
-import androidx.compose.ui.text.style.TextAlign
-import androidx.compose.ui.text.style.TextDecoration
-import androidx.compose.ui.text.style.TextOverflow
-import androidx.compose.ui.unit.Dp
-import androidx.compose.ui.unit.IntSize
+import androidx.compose.ui.tooling.preview.Preview
+import androidx.compose.ui.tooling.preview.PreviewParameter
import androidx.compose.ui.unit.dp
-import androidx.compose.ui.unit.sp
-import coil3.compose.AsyncImage
import com.app.assistant.R
-import com.app.assistant.model.Conversation
+import com.app.assistant.llm.LlmProvider
import com.app.assistant.ui.theme.AssistantTheme
-import com.app.assistant.util.Category
import com.app.assistant.viewmodel.MainViewModel
-import kotlinx.coroutines.currentCoroutineContext
-import org.commonmark.node.Text as CText
-import kotlinx.coroutines.flow.collectLatest
-import kotlinx.coroutines.isActive
+import com.app.assistant.viewmodel.SettingsViewModel
import kotlinx.coroutines.launch
-import org.commonmark.node.BlockQuote
-import org.commonmark.node.BulletList
-import org.commonmark.node.Code
-import org.commonmark.node.Emphasis
-import org.commonmark.node.FencedCodeBlock
-import org.commonmark.node.HardLineBreak
-import org.commonmark.node.Heading
-import org.commonmark.node.IndentedCodeBlock
-import org.commonmark.node.Link
-import org.commonmark.node.ListItem
-import org.commonmark.node.Node
-import org.commonmark.node.OrderedList
-import org.commonmark.node.Paragraph
-import org.commonmark.node.SoftLineBreak
-import org.commonmark.node.StrongEmphasis
-import org.commonmark.node.ThematicBreak
-import org.commonmark.parser.Parser
-import kotlin.math.hypot
+
+import android.Manifest
+import android.content.pm.PackageManager
+import android.widget.Toast
+import androidx.core.content.ContextCompat
+import kotlinx.coroutines.flow.collectLatest
+import com.app.assistant.model.Group
+import com.app.assistant.model.Conversation
+import androidx.compose.ui.focus.FocusManager
+import androidx.compose.ui.platform.SoftwareKeyboardController
@OptIn(ExperimentalMaterial3Api::class)
@Composable
-fun SetupUI(viewModel: MainViewModel) {
-
+fun SetupUI(
+ viewModel: MainViewModel,
+ settingsViewModel: SettingsViewModel,
+ onToggleVisionMode: () -> Unit,
+ onToggleScreenMode: () -> Unit
+) {
AssistantTheme {
- var showCopyIcon by remember { mutableStateOf(false) }
- var selectedItemIndex by remember { mutableStateOf(null) }
- val clipboardManager = LocalClipboardManager.current
- var clearShowDialog by remember { mutableStateOf(false) }
- var deleteShowDialog by remember { mutableStateOf(false) }
- var showSettingsDialog by remember { mutableStateOf(false) }
-
- val sheetState = rememberModalBottomSheetState()
- var showBottomSheet by viewModel.showBottomSheet
-
val isCustomUI by viewModel.isCustomUI.collectAsState()
val isCustomUIHalfPage by viewModel.isCustomUIHalfPage.collectAsState()
- val drawerState = remember(isCustomUI) {
- DrawerState(DrawerValue.Closed)
- }
- val scope = rememberCoroutineScope()
+ val isVisionModeActive by viewModel.isVisionModeActive.collectAsState()
+ val isScreenModeActive by viewModel.isScreenModeActive.collectAsState()
val groupList by viewModel.groupList.collectAsState(initial = emptyList())
+ val chatList = viewModel.chatList
+ val isSpeaking by viewModel.isSpeaking.collectAsState()
+ val isListening by viewModel.isListening.collectAsState()
+ val isVoiceProcessing by viewModel.isVoiceProcessing.collectAsState()
+ val isHandsFree by viewModel.isHandsFreeModeActive.collectAsState()
+ val isMicMuted by viewModel.isMicMuted.collectAsState()
+ val question by viewModel.question.collectAsState()
+ val isTranslationEnabled by settingsViewModel.isTranslationEnabled.collectAsState()
+ val isDeletingChat by viewModel.isDeletingChat.collectAsState()
+ val audioAmplitude by viewModel.audioAmplitude.collectAsState()
- val screenHeight = LocalConfiguration.current.screenHeightDp.dp
- val backPressedDispatcher = LocalOnBackPressedDispatcherOwner.current?.onBackPressedDispatcher
- ModalNavigationDrawer(
- drawerState = drawerState,
- drawerContent = {
- if (isCustomUI) null else {
- ModalDrawerSheet {
- Text("Chats", modifier = Modifier.padding(16.dp))
- Column(
- modifier = Modifier
- .padding(horizontal = 8.dp)
- .weight(1f)
- .verticalScroll(rememberScrollState())
- ) {
- groupList.forEach { group ->
- NavigationDrawerItem(
- label = { Text(text = group.title) },
- selected = false,
- onClick = {
- viewModel.loadMessagesFromGroup(group.groupId)
- scope.launch { drawerState.close() }
- }
- )
- }
- }
- HorizontalDivider(modifier = Modifier.padding(vertical = 8.dp))
- NavigationDrawerItem(
- label = { Text(text = "Settings") },
- icon = {
- Icon(
- painter = (painterResource(id = R.drawable.ic_settings)),
- contentDescription = "Settings"
- )
- },
- selected = false,
- onClick = {
- showSettingsDialog = true
- scope.launch { drawerState.close() }
- }
- )
- NavigationDrawerItem(
- label = { Text(text = "Start new chat") },
- icon = {
- Icon(
- painter = (painterResource(id = R.drawable.ic_add_new_chat)),
- contentDescription = "Start new chat"
- )
- },
- selected = false,
- onClick = {
- viewModel.newChat()
- scope.launch { drawerState.close() }
- }
- )
- }
+ var currentScreen by remember { mutableStateOf("chat") }
+
+ val selectedAttachments by viewModel.selectedAttachments.collectAsState()
+ val isImageSupported = remember(currentScreen) { viewModel.settingsRepository.getIsImageSupported() }
+ val isAudioSupported = remember(currentScreen) { viewModel.settingsRepository.getIsAudioSupported() }
+ val isVideoSupported = remember(currentScreen) { viewModel.settingsRepository.getIsVideoSupported() }
+ val isDocumentSupported = remember(currentScreen) { viewModel.settingsRepository.getIsDocumentSupported() }
+
+ val pickerLauncher = androidx.activity.compose.rememberLauncherForActivityResult(
+ contract = androidx.activity.result.contract.ActivityResultContracts.GetContent()
+ ) { uri ->
+ uri?.let { viewModel.addSelectedAttachment(it) }
+ }
+
+ val context = LocalContext.current
+
+ var tempCameraUri by remember { mutableStateOf(null) }
+ val cameraLauncher = androidx.activity.compose.rememberLauncherForActivityResult(
+ contract = androidx.activity.result.contract.ActivityResultContracts.TakePicture()
+ ) { success ->
+ if (success) {
+ tempCameraUri?.let { uri ->
+ viewModel.addSelectedAttachment(uri)
}
}
- ) {
- val scaffoldContent = @Composable{
- Scaffold(
- containerColor = if(isCustomUIHalfPage) Color.Transparent else MaterialTheme.colorScheme.background,
- modifier = if (isCustomUI) {
+ }
+
+ fun launchCamera() {
+ try {
+ val file = java.io.File(context.cacheDir, "camera_capture_${System.currentTimeMillis()}.jpg")
+ val authority = "${context.packageName}.fileprovider"
+ val uri = androidx.core.content.FileProvider.getUriForFile(context, authority, file)
+ tempCameraUri = uri
+ cameraLauncher.launch(uri)
+ } catch (e: Exception) {
+ android.util.Log.e("ChatScreen", "Failed to launch camera", e)
+ Toast.makeText(context, "Failed to launch camera", Toast.LENGTH_SHORT).show()
+ }
+ }
+
+ val cameraPermissionLauncher = androidx.activity.compose.rememberLauncherForActivityResult(
+ contract = androidx.activity.result.contract.ActivityResultContracts.RequestPermission()
+ ) { isGranted ->
+ if (isGranted) {
+ launchCamera()
+ } else {
+ Toast.makeText(context, "Camera permission is required to take photos", Toast.LENGTH_SHORT).show()
+ }
+ }
+
+ LaunchedEffect(Unit) {
+ viewModel.showToastEvent.collectLatest { message: String ->
+ Toast.makeText(context, message, Toast.LENGTH_SHORT).show()
+ }
+ }
+
+ val clipboardManager = LocalClipboardManager.current
+
+ if (currentScreen == "settings") {
+ SettingsScreen(
+ settingsViewModel = settingsViewModel,
+ onBack = { currentScreen = "chat" },
+ onTranslationEnabledChange = { enabled ->
+ settingsViewModel.updateTranslationEnabled(enabled)
+ },
+ onLanguageSelected = { langCode ->
+ settingsViewModel.updateActiveLanguageCode(langCode)
+ }
+ )
+ } else {
+ ChatScreenContent(
+ isCustomUI = isCustomUI,
+ isCustomUIHalfPage = isCustomUIHalfPage,
+ groupList = groupList,
+ chatList = chatList,
+ isSpeaking = isSpeaking,
+ isListening = isListening,
+ audioAmplitude = audioAmplitude,
+ isHandsFree = isHandsFree,
+ isVoiceProcessing = isVoiceProcessing,
+ question = question,
+ isTranslationEnabled = isTranslationEnabled,
+ isDeletingChat = isDeletingChat,
+ onDeleteGroupClick = { viewModel.deleteGroup(it) },
+ onGroupClick = { viewModel.loadMessagesFromGroup(it) },
+ onSettingsClick = { currentScreen = "settings" },
+ onNewChatClick = { viewModel.newChat() },
+ onCopyClick = { index ->
+ val conversation = viewModel.chatList.getOrNull(index)
+ val textToCopy = conversation?.text ?: ""
+ clipboardManager.setText(AnnotatedString(textToCopy))
+ },
+ onDeleteClick = { viewModel.deleteMessage(it) },
+ onClearChatClick = { viewModel.clearBoxes() },
+ onMenuClick = { /* Handled inside drawer state toggling */ },
+ onDragEnd = { viewModel.expandToFullScreen() },
+ onQuestionChange = { viewModel.setQuestion(it) },
+ onStopSpeaking = { viewModel.stopTextToSpeech() },
+ onStartListening = {
+ viewModel.startSpeechRecognition()
+ },
+ onProcessQuestion = { fm, kc, isVoice ->
+ viewModel.processQuestion(fm, kc, isVoice)
+ },
+ onBackPressed = { /* Handled in parent context back handler */ },
+ selectedAttachments = selectedAttachments,
+ onRemoveAttachment = { viewModel.removeSelectedAttachment(it) },
+ onAttachClick = { type ->
+ if (type == "camera") {
+ val hasCameraPermission = ContextCompat.checkSelfPermission(
+ context,
+ Manifest.permission.CAMERA
+ ) == PackageManager.PERMISSION_GRANTED
+ if (hasCameraPermission) {
+ launchCamera()
+ } else {
+ cameraPermissionLauncher.launch(Manifest.permission.CAMERA)
+ }
+ } else {
+ pickerLauncher.launch(type)
+ }
+ },
+ isImageSupported = isImageSupported,
+ isAudioSupported = isAudioSupported,
+ isVideoSupported = isVideoSupported,
+ isDocumentSupported = isDocumentSupported,
+ onToggleHandsFree = { viewModel.toggleHandsFreeMode() },
+ onExitHandsFree = { viewModel.setHandsFreeModeActive(false) },
+ isMicMuted = isMicMuted,
+ onToggleMicMute = { viewModel.toggleMicMute() },
+ isVisionModeActive = isVisionModeActive,
+ onToggleVisionMode = onToggleVisionMode,
+ isScreenModeActive = isScreenModeActive,
+ onToggleScreenMode = onToggleScreenMode
+ )
+ }
+ }
+}
+
+@OptIn(ExperimentalMaterial3Api::class)
+@Composable
+fun ChatScreenContent(
+ isCustomUI: Boolean,
+ isCustomUIHalfPage: Boolean,
+ groupList: List,
+ chatList: List,
+ isSpeaking: Boolean,
+ isListening: Boolean,
+ audioAmplitude: Float = 0f,
+ question: String,
+ isTranslationEnabled: Boolean,
+ isDeletingChat: Boolean,
+ onGroupClick: (Long) -> Unit,
+ onSettingsClick: () -> Unit,
+ onNewChatClick: () -> Unit,
+ onCopyClick: (Int) -> Unit,
+ onDeleteClick: (Int) -> Unit,
+ onClearChatClick: () -> Unit,
+ onMenuClick: () -> Unit,
+ onDragEnd: () -> Unit,
+ onQuestionChange: (String) -> Unit,
+ onStopSpeaking: () -> Unit,
+ onStartListening: () -> Unit,
+ onProcessQuestion: (FocusManager, SoftwareKeyboardController, Boolean) -> Unit,
+ onBackPressed: () -> Unit,
+ onDeleteGroupClick: (Long) -> Unit,
+ selectedAttachments: List = emptyList(),
+ onRemoveAttachment: (com.app.assistant.model.Attachment) -> Unit = {},
+ onAttachClick: (String) -> Unit = {},
+ isImageSupported: Boolean = false,
+ isAudioSupported: Boolean = false,
+ isVideoSupported: Boolean = false,
+ isDocumentSupported: Boolean = false,
+ isHandsFree: Boolean = false,
+ onToggleHandsFree: () -> Unit = {},
+ onExitHandsFree: () -> Unit = {},
+ isVoiceProcessing: Boolean = false,
+ isMicMuted: Boolean = false,
+ onToggleMicMute: () -> Unit = {},
+ isVisionModeActive: Boolean = false,
+ onToggleVisionMode: () -> Unit = {},
+ isScreenModeActive: Boolean = false,
+ onToggleScreenMode: () -> Unit = {},
+) {
+ var showCopyIcon by remember { mutableStateOf(false) }
+ var selectedItemIndex by remember { mutableStateOf(null) }
+ val clipboardManager = LocalClipboardManager.current
+ var clearShowDialog by remember { mutableStateOf(false) }
+ var deleteShowDialog by remember { mutableStateOf(false) }
+
+ val scope = rememberCoroutineScope()
+ val screenHeight = LocalConfiguration.current.screenHeightDp.dp
+ val backPressedDispatcher = LocalOnBackPressedDispatcherOwner.current?.onBackPressedDispatcher
+
+ val drawerState = remember(isCustomUI) {
+ DrawerState(DrawerValue.Closed)
+ }
+
+ ModalNavigationDrawer(
+ drawerState = drawerState,
+ drawerContent = {
+ if (!isCustomUI) {
+ ChatDrawerContent(
+ groupList = groupList,
+ isDeletingChat = isDeletingChat,
+ onGroupClick = { groupId ->
+ onGroupClick(groupId)
+ scope.launch { drawerState.close() }
+ },
+ onSettingsClick = {
+ onSettingsClick()
+ scope.launch { drawerState.close() }
+ },
+ onNewChatClick = {
+ onNewChatClick()
+ scope.launch { drawerState.close() }
+ },
+ onDeleteGroupClick = onDeleteGroupClick
+ )
+ }
+ },
+ ) {
+ val scaffoldContent = @Composable {
+ Scaffold(
+ containerColor = if (isCustomUIHalfPage) Color.Transparent else MaterialTheme.colorScheme.background,
+ modifier =
+ if (isCustomUI) {
Modifier
.height(screenHeight / 2)
.padding(16.dp, 16.dp, 16.dp, 8.dp)
@@ -250,141 +370,95 @@ fun SetupUI(viewModel: MainViewModel) {
} else {
Modifier
},
- topBar = {
- if (!isCustomUI) {
- TopAppBar(
- navigationIcon = {
- IconButton(onClick = {
- scope.launch {
- drawerState.apply {
- if (isClosed) open() else close()
- }
- }
- }) {
- Icon(
- painter = (painterResource(id = R.drawable.ic_menu)),
- contentDescription = "Menu"
- )
- }
- },
- title = { Text("") },
- actions = {
- if (showCopyIcon) {
- IconButton(onClick = {
- selectedItemIndex?.let { index ->
- val textToCopy = when {
- viewModel.getIsTranslationEnabled() -> viewModel.chatList.getOrNull(
- index
- )?.translatedText
-
- else -> viewModel.chatList.getOrNull(index)?.englishText
- } ?: ""
- clipboardManager.setText(AnnotatedString(textToCopy))
- selectedItemIndex = null
- }
- }) {
- Icon(
- painter = painterResource(id = R.drawable.ic_content_copy),
- contentDescription = "Copy"
- )
- }
- IconButton(onClick = {
- deleteShowDialog = true
- }) {
- Icon(
- painter = painterResource(id = R.drawable.ic_delete),
- contentDescription = "Delete"
- )
- }
- }
- IconButton(onClick = {
- showBottomSheet = true
- }) {
- Icon(
- painter = (painterResource(id = R.drawable.ic_translate)),
- contentDescription = "Translate"
- )
- }
- if (viewModel.chatList.isNotEmpty()) {
- IconButton(onClick = { clearShowDialog = true }) {
- Icon(
- painter = (painterResource(id = R.drawable.ic_restart)),
- contentDescription = "Restart Chat"
- )
- }
+ topBar = {
+ if (!isCustomUI) {
+ ChatTopAppBar(
+ showCopyIcon = showCopyIcon,
+ onCopyClick = {
+ selectedItemIndex?.let { index ->
+ onCopyClick(index)
+ selectedItemIndex = null
+ }
+ },
+ onDeleteClick = { deleteShowDialog = true },
+ chatListNotEmpty = chatList.isNotEmpty(),
+ onClearChatClick = { clearShowDialog = true },
+ onMenuClick = {
+ scope.launch {
+ drawerState.apply {
+ if (isClosed) open() else close()
}
}
- )
- }
- if(!isCustomUIHalfPage && isCustomUI){
- Box(
- modifier = Modifier
- .fillMaxWidth()
- .height(24.dp)
- .then(
- Modifier
- .pointerInput(Unit) {
- detectDragGestures(
- onDragStart = { /* Optional: trigger on drag start */ },
- onDragEnd = {
- viewModel.expandToFullScreen()
- },
- onDrag = { change, dragAmount ->
- //track dragAmount if needed
- }
- )
- })
- .clickable { viewModel.expandToFullScreen() },
- contentAlignment = Alignment.Center
- ) {
- Box(
- modifier = Modifier
- .width(36.dp)
- .height(4.dp)
- .background(
- color = Color.Gray,
- shape = RoundedCornerShape(2.dp)
- )
- )
}
-
- }
+ )
}
- ) { innerPadding ->
- MyLayout(
- modifier = Modifier.padding(innerPadding),
- onShowCopyIconChange = { newValue ->
- showCopyIcon = newValue
- },
- selectedItemIndex = selectedItemIndex,
- onSelectedItemChange = { newIndex -> selectedItemIndex = newIndex },
- viewModel = viewModel,
- isCustomUI = isCustomUI
- )
- }
+ if (!isCustomUIHalfPage && isCustomUI) {
+ CustomUiDragHandle(
+ onDragEnd = onDragEnd
+ )
+ }
+ },
+ ) { innerPadding ->
+ ChatLayout(
+ modifier = Modifier.padding(innerPadding),
+ onShowCopyIconChange = { newValue ->
+ showCopyIcon = newValue
+ },
+ selectedItemIndex = selectedItemIndex,
+ onSelectedItemChange = { newIndex -> selectedItemIndex = newIndex },
+ isCustomUI = isCustomUI,
+ chatList = chatList,
+ isSpeaking = isSpeaking,
+ isListening = isListening,
+ audioAmplitude = audioAmplitude,
+ question = question,
+ onQuestionChange = onQuestionChange,
+ onStopSpeaking = onStopSpeaking,
+ onStartListening = onStartListening,
+ onProcessQuestion = onProcessQuestion,
+ isTranslateEnabled = isTranslationEnabled,
+ selectedAttachments = selectedAttachments,
+ onRemoveAttachment = onRemoveAttachment,
+ onAttachClick = onAttachClick,
+ isImageSupported = isImageSupported,
+ isAudioSupported = isAudioSupported,
+ isVideoSupported = isVideoSupported,
+ isDocumentSupported = isDocumentSupported,
+ isHandsFree = isHandsFree,
+ onToggleHandsFree = onToggleHandsFree,
+ onExitHandsFree = onExitHandsFree,
+ isVoiceProcessing = isVoiceProcessing,
+ isMicMuted = isMicMuted,
+ onToggleMicMute = onToggleMicMute,
+ isVisionModeActive = isVisionModeActive,
+ onToggleVisionMode = onToggleVisionMode,
+ isScreenModeActive = isScreenModeActive,
+ onToggleScreenMode = onToggleScreenMode,
+ )
+ }
+ }
+
+ if (isCustomUI) {
+ var tapped by remember { mutableStateOf(false) }
+
+ LaunchedEffect(Unit) {
+ tapped = false
}
- if (isCustomUI) {
- var tapped by remember { mutableStateOf(false) }
- val backPressedDispatcher = LocalOnBackPressedDispatcherOwner.current?.onBackPressedDispatcher
-
- LaunchedEffect(Unit) {
- tapped = false
- }
-
- val animatedBoxAlpha by animateFloatAsState(
- targetValue = if (tapped) 0.0f else 0.3f,
- animationSpec = tween(durationMillis = 100), // Quick animation
- label = "boxBackgroundAlpha",
- finishedListener = {
- if (tapped) {
- backPressedDispatcher?.onBackPressed()
- }
+ val animatedBoxAlpha by animateFloatAsState(
+ targetValue = if (tapped) 0.0f else 0.3f,
+ animationSpec = tween(durationMillis = 100), // Quick animation
+ label = "boxBackgroundAlpha",
+ finishedListener = {
+ if (tapped) {
+ backPressedDispatcher?.onBackPressed()
}
- )
+ },
+ )
- Box(
- modifier = Modifier
+ Box(
+ modifier =
+ Modifier
.fillMaxSize()
.background(Color.Black.copy(alpha = animatedBoxAlpha))
.imePadding()
@@ -393,950 +467,98 @@ fun SetupUI(viewModel: MainViewModel) {
enabled = !tapped,
onClick = {
tapped = true
- }
+ },
),
- contentAlignment = Alignment.BottomCenter
- ) {
- scaffoldContent()
- }
- }
- else{
+ contentAlignment = Alignment.BottomCenter,
+ ) {
scaffoldContent()
}
- }
-
- if (showBottomSheet) {
- var searchQuery by remember { mutableStateOf("") }
- val filteredItems = viewModel.languages.filter {it.first.contains(searchQuery, true)}.map { it }
- val isLanguageLoading by viewModel.isLanguageLoading.collectAsState()
- val isTranslationEnabled by viewModel.isTranslationEnabled.collectAsState()
-
- ModalBottomSheet(
- onDismissRequest = {
- showBottomSheet = false
- },
- sheetState = sheetState
- ) {
- Column(
- modifier = Modifier.fillMaxSize(),
- ) {
- Row(modifier = Modifier
- .fillMaxWidth()
- .padding(16.dp, 0.dp, 16.dp, 16.dp)) {
- Text(
- modifier = Modifier
- .weight(1f)
- .align(Alignment.CenterVertically),
- text = "Chat in different languages"
- )
- Switch(checked = isTranslationEnabled,
- onCheckedChange = { viewModel.updateTranslationEnabled(it) },
- modifier = Modifier.align(
- Alignment.CenterVertically))
- }
-
-
- TextField(
- modifier = Modifier
- .fillMaxWidth()
- .padding(0.dp)
- .imePadding(),
- value = searchQuery,
- placeholder = { Text("Search...") },
- singleLine = true,
- onValueChange = { searchQuery = it },
- colors = TextFieldDefaults.colors(
- focusedIndicatorColor = Color.Transparent,
- unfocusedIndicatorColor = Color.Transparent,
- disabledIndicatorColor = Color.Transparent
- ),
- enabled = isTranslationEnabled
- )
-
- if(isLanguageLoading)
- LinearProgressIndicator(modifier = Modifier.fillMaxWidth())
-
- LazyColumn(
- modifier = Modifier
- .fillMaxSize()
- .padding(0.dp, 16.dp, 0.dp, 0.dp),
- ) {
- items(items = filteredItems) { lang ->
- Text(
- text = lang.first,
- modifier = Modifier
- .fillMaxWidth()
- .clickable(enabled = isTranslationEnabled) {
- if (isTranslationEnabled) {
- viewModel.onItemSelected(lang.second)
- }
- }
- .padding(16.dp),
- )
- }
- }
- }
- }
- }
-
- if (clearShowDialog) {
- AlertDialog(
- onDismissRequest = { clearShowDialog = false },
- text = { Text("Are you sure you want to clear this chat from history?") },
- confirmButton = {
- TextButton(onClick = {
- viewModel.clearBoxes()
- clearShowDialog = false
- }) {
- Text("Yes")
- }
- },
- dismissButton = {
- TextButton(onClick = { clearShowDialog = false }) {
- Text("No")
- }
- }
- )
- }
-
- if(deleteShowDialog){
- AlertDialog(
- onDismissRequest = {
- deleteShowDialog = false
- selectedItemIndex = null
- },
- text = { Text("Delete this message?") },
- confirmButton = {
- TextButton(onClick = {
- selectedItemIndex?.let { index ->
- viewModel.deleteMessage(index)
- }
- deleteShowDialog = false
- selectedItemIndex = null
- }) {
- Text("Yes")
- }
- },
- dismissButton = {
- TextButton(onClick = {
- deleteShowDialog = false
- selectedItemIndex = null
- }) {
- Text("No")
- }
- }
- )
- }
-
- if (showSettingsDialog) {
- val youtubeKey = viewModel.loadYoutubeKey() ?: ""
- val chatKey = viewModel.loadChatKey() ?: ""
-
- SettingsDialog(
- initialYoutubeKey = youtubeKey,
- initialChatKey = chatKey,
- onDismiss = { showSettingsDialog = false },
- onSave = { ytKey, chKey ->
- viewModel.saveKeys(ytKey, chKey)
- showSettingsDialog = false
- }
- )
- }
-
- BackHandler(enabled = selectedItemIndex != null) {
- selectedItemIndex = null // Clear selection instead of handling system back
- }
- }
-}
-
-@Composable
-fun SettingsDialog(
- initialYoutubeKey: String,
- initialChatKey: String,
- onDismiss: () -> Unit,
- onSave: (String, String) -> Unit
-) {
- var apiKey1 by rememberSaveable { mutableStateOf(initialYoutubeKey) }
- var apiKey2 by rememberSaveable { mutableStateOf(initialChatKey) }
-
- val context = LocalContext.current
- val packageInfo = context.packageManager.getPackageInfo(context.packageName, 0)
- val versionName = packageInfo.versionName
-
- AlertDialog(
- onDismissRequest = onDismiss,
- title = { Text("Settings") },
- text = {
- Column {
- TextField(
- value = apiKey1,
- onValueChange = { apiKey1 = it },
- label = { Text("YouTube API Key") }
- )
- Spacer(modifier = Modifier.height(8.dp))
- TextField(
- value = apiKey2,
- onValueChange = { apiKey2 = it },
- label = { Text("Chat API Key") }
- )
- Spacer(modifier = Modifier.height(16.dp))
- Text("App Version: $versionName")
- }
- },
- confirmButton = {
- Button(onClick = { onSave(apiKey1, apiKey2) }) {
- Text("Save")
- }
- },
- dismissButton = {
- Button(onClick = onDismiss) {
- Text("Cancel")
- }
- }
- )
-}
-
-@Composable
-fun MyLayout(
- modifier: Modifier = Modifier,
- onShowCopyIconChange: (Boolean) -> Unit,
- selectedItemIndex: Int?,
- onSelectedItemChange: (Int) -> Unit,
- viewModel: MainViewModel,
- isCustomUI: Boolean
-) {
- val boxes = viewModel.chatList
- val keyboardController = LocalSoftwareKeyboardController.current
- val focusManager = LocalFocusManager.current
- val scrollState = rememberLazyListState()
-
- val context = LocalContext.current
- val vibrator = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
- context.getSystemService(VibratorManager::class.java)?.defaultVibrator
- } else {
- @Suppress("DEPRECATION")
- context.getSystemService(Vibrator::class.java)
- }
-
- LaunchedEffect(Unit) {
- viewModel.showToastEvent.collectLatest { message: String ->
- Toast.makeText(context, message, Toast.LENGTH_SHORT).show()
- }
- }
-
- LaunchedEffect(selectedItemIndex) {
- onShowCopyIconChange(selectedItemIndex != null)
- if (selectedItemIndex != null) {
- if (vibrator?.hasVibrator() == true) {
- if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
- vibrator.vibrate(VibrationEffect.createOneShot(50, VibrationEffect.DEFAULT_AMPLITUDE))
- } else {
- @Suppress("DEPRECATION")
- vibrator.vibrate(50)
- }
- }
- }
- }
-
- Column(modifier = modifier.fillMaxSize()) {
- LazyColumn(
- modifier = Modifier
- .weight(1f)
- .padding(0.dp, 0.dp, 0.dp, 0.dp),
- verticalArrangement = if (isCustomUI)
- Arrangement.spacedBy(8.dp, Alignment.Bottom)
- else
- Arrangement.spacedBy(8.dp),
- state = scrollState,
- ) {
- itemsIndexed(boxes) { index, conversation ->
- ConversationItem(
- conversation = conversation,
- index = index,
- isSelected = index == selectedItemIndex,
- onLongClick = { newIndex -> onSelectedItemChange(newIndex) },
- viewModel = viewModel
- )
- }
- }
-
- if (keyboardController != null) {
- UserInputField(
- focusManager = focusManager,
- keyboardController = keyboardController,
- viewModel = viewModel
- )
- }
- }
-
- LaunchedEffect(boxes.size) {
- if (boxes.isNotEmpty()) {
- scrollState.animateScrollToItem(boxes.size - 1)
- }
- }
-}
-
-@Composable
-private fun UserInputField(
- focusManager: FocusManager,
- keyboardController: SoftwareKeyboardController,
- viewModel: MainViewModel
-) {
- val isSpeaking by viewModel.isSpeaking.collectAsState()
- val isListening by viewModel.isListening.collectAsState()
-
- val containerColor = TextFieldDefaults.colors().unfocusedContainerColor
- val rippleColor = adjustContrast(containerColor)
-
- // State for current radius + alpha
- val rippleRadius = remember { Animatable(0f) }
- val rippleAlpha = remember { Animatable(0f) }
-
- // Size of TextField (for max ripple radius)
- var textFieldSize by remember { mutableStateOf(IntSize.Zero) }
-
- LaunchedEffect(isListening, textFieldSize) {
- if (isListening && textFieldSize != IntSize.Zero) {
- val maxRadius = hypot(
- textFieldSize.width / 2f,
- textFieldSize.height / 2f
- )
- while (currentCoroutineContext().isActive) { // keep running until LaunchedEffect cancels
- rippleRadius.snapTo(0f)
- rippleAlpha.snapTo(0.5f)
-
- // animate simultaneously
- launch {
- rippleRadius.animateTo(
- targetValue = maxRadius,
- animationSpec = tween(1000, easing = LinearEasing)
- )
- }
- rippleAlpha.animateTo(
- targetValue = 0f,
- animationSpec = tween(1000, easing = LinearEasing)
- )
- }
} else {
- rippleRadius.snapTo(0f)
- rippleAlpha.snapTo(0f)
+ scaffoldContent()
}
}
+ // Translation Bottom Sheet removed (moved to Settings)
- Box(
- modifier = Modifier
- .fillMaxWidth()
- .imePadding()
- .padding(8.dp)
- .clip(RoundedCornerShape(20)) // clip ripple inside
- .onGloballyPositioned { coords -> textFieldSize = coords.size }
- ) {
- TextField(
- modifier = Modifier
- .fillMaxWidth(),
- shape = RoundedCornerShape(20),
- value = viewModel.question.value,
- placeholder = { Text("Type here...") },
- maxLines = 2,
- onValueChange = { viewModel.question.value = it },
- colors = TextFieldDefaults.colors(
- focusedIndicatorColor = Color.Transparent,
- unfocusedIndicatorColor = Color.Transparent,
- disabledIndicatorColor = Color.Transparent
- ),
- trailingIcon = {
- Row(horizontalArrangement = Arrangement.End) {
- val context = LocalContext.current
- if (isSpeaking) {
- IconButton(onClick = {
- if (viewModel.textToSpeech.isSpeaking) {
- viewModel.stopTextToSpeech()
- }
- }) {
- Icon(
- painter = painterResource(id = R.drawable.ic_stop),
- contentDescription = "Stop"
- )
- }
- } else {
- IconButton(onClick = {
- viewModel.startSpeechToText(
- context,
- onResult = { recognizedText ->
- viewModel.question.value = recognizedText
- if (viewModel.question.value.isNotEmpty()) {
- viewModel.processQuestion(
- focusManager,
- keyboardController,
- context,
- true
- )
- }
- },
- onPartialResult = { recognizedText ->
- viewModel.question.value = recognizedText
- }
- )
- }) {
- Icon(
- painter = painterResource(id = R.drawable.ic_mic),
- contentDescription = "Mic"
- )
- }
- }
- IconButton(onClick = {
- if (viewModel.question.value.isNotEmpty()) {
- viewModel.processQuestion(
- focusManager,
- keyboardController,
- context,
- false
- )
- }
- }) {
- Icon(
- imageVector = Icons.AutoMirrored.Filled.Send,
- contentDescription = "Send"
- )
- }
- }
- }
- )
-
- Canvas(
- modifier = Modifier
- .matchParentSize()
- .clip(RoundedCornerShape(20))
- ) {
- if (rippleAlpha.value > 0f) {
- drawCircle(
- color = rippleColor.copy(alpha = rippleAlpha.value),
- radius = rippleRadius.value,
- center = center
- )
- }
- }
- }
-
-}
-
-fun adjustContrast(color: Color, factor: Float = 0.15f): Color {
- val isDark = color.luminance() < 0.5
- return if (isDark) {
- // lighten in dark mode
- Color(
- red = (color.red + factor).coerceAtMost(1f),
- green = (color.green + factor).coerceAtMost(1f),
- blue = (color.blue + factor).coerceAtMost(1f),
- alpha = color.alpha
- )
- } else {
- // darken in light mode
- Color(
- red = (color.red - factor).coerceAtLeast(0f),
- green = (color.green - factor).coerceAtLeast(0f),
- blue = (color.blue - factor).coerceAtLeast(0f),
- alpha = color.alpha
- )
- }
-}
-
-
-
-@Composable
-fun ConversationItem(
- conversation: Conversation,
- index: Int,
- isSelected: Boolean,
- onLongClick: (Int) -> Unit,
- viewModel: MainViewModel,
- backgroundColor: Color = if (conversation.isMe) MaterialTheme.colorScheme.secondary else MaterialTheme.colorScheme.surfaceContainerHighest,
- cornerRadius: Dp = 20.dp
-) {
- val uriHandler = LocalUriHandler.current
- val isTranslateEnabled = viewModel.getIsTranslationEnabled()
- Column(
- modifier = Modifier.fillMaxWidth()
- ) {
- Row(
- modifier = Modifier
- .fillMaxWidth()
- .padding(horizontal = 8.dp),
- horizontalArrangement = if (conversation.isMe) Arrangement.Start else Arrangement.End
- ) {
- if (conversation.isMe)
- Spacer(modifier = Modifier.weight(1f))
- Card(
- Modifier
- .wrapContentWidth()
- .widthIn(max = LocalConfiguration.current.screenWidthDp.dp * 0.85f),
- shape = RoundedCornerShape(cornerRadius),
- colors = CardDefaults.cardColors(containerColor = if (isSelected) MaterialTheme.colorScheme.secondaryContainer else backgroundColor)
- ) {
- if (conversation.isLoading) {
- LoadingDots() // Show loading animation when isLoading is true
- } else {
- if(conversation.isMe)
- OtherCard(conversation, onLongClick, index, isTranslateEnabled)
- else {
- when (conversation.category) {
- Category.CALL.name -> {
- MakeCall(uriHandler, conversation, isTranslateEnabled)
- }
-
- Category.OTHER.name -> {
- OtherCard(conversation, onLongClick, index, isTranslateEnabled)
- }
-
- Category.SETTINGS.name -> {
- // TODO
- OtherCard(conversation, onLongClick, index, isTranslateEnabled)
- }
-
- Category.SONGS.name -> {
- PlaySong(uriHandler, conversation, isTranslateEnabled)
- }
-
- Category.NAVIGATION.name -> {
- StartNavigation(uriHandler, conversation, isTranslateEnabled)
- }
-
- Category.WEATHER.name -> {
- ShowWeather(uriHandler, conversation, isTranslateEnabled)
- }
-
- Category.REMINDER.name -> {
- OtherCard(conversation, onLongClick, index, isTranslateEnabled)
- }
-
- Category.ALARM.name -> {
- OtherCard(conversation, onLongClick, index, isTranslateEnabled)
- }
-
- else -> {
- OtherCard(conversation, onLongClick, index, isTranslateEnabled)
- }
- }
- }
- }
- }
- if (!conversation.isMe)
- Spacer(modifier = Modifier.weight(1f))
- }
-
- if (conversation.isMe) {
- Text(
- text = conversation.category,
- modifier = Modifier
- .padding(0.dp, 0.dp, 20.dp, 0.dp)
- .fillMaxWidth(),
- maxLines = 1,
- overflow = TextOverflow.Ellipsis,
- color = MaterialTheme.colorScheme.secondary,
- textAlign = TextAlign.End
- )
- }
- }
-}
-
-@Composable
-private fun StartNavigation(
- uriHandler: UriHandler,
- conversation: Conversation,
- isTranslateEnabled: Boolean
-) {
- Row(
- modifier = Modifier
- .padding(8.dp)
- .clickable {
- uriHandler.openUri(conversation.navigationURI.toString())
- }, horizontalArrangement = Arrangement.spacedBy(8.dp)
- ) {
- Box(
- modifier = Modifier
- .background(
- shape = CircleShape,
- color = MaterialTheme.colorScheme.secondaryContainer
- ),
- ) {
- Icon(
- modifier = Modifier.padding(8.dp),
- imageVector = Icons.Default.LocationOn,
- contentDescription = "Navigation",
- )
- }
-
- MarkdownText(
- modifier = Modifier
- .align(alignment = Alignment.CenterVertically)
- .clickable {
- uriHandler.openUri(conversation.navigationURI.toString())
- }, markdown = if(isTranslateEnabled) conversation.translatedText else conversation.englishText
- )
- }
-}
-
-@Composable
-private fun PlaySong(
- uriHandler: UriHandler,
- conversation: Conversation,
- isTranslateEnabled: Boolean
-) {
- AsyncImage(
- modifier = Modifier
- .clickable {
- uriHandler.openUri(conversation.navigationURI.toString())
+ if (clearShowDialog) {
+ ClearChatDialog(
+ onConfirm = {
+ onClearChatClick()
+ clearShowDialog = false
},
- model = conversation.contentURL,
- contentDescription = if(isTranslateEnabled) conversation.translatedText else conversation.englishText
- )
+ onDismiss = { clearShowDialog = false }
+ )
+ }
+
+ if (deleteShowDialog) {
+ DeleteMessageDialog(
+ onConfirm = {
+ selectedItemIndex?.let { index ->
+ onDeleteClick(index)
+ }
+ deleteShowDialog = false
+ selectedItemIndex = null
+ },
+ onDismiss = {
+ deleteShowDialog = false
+ selectedItemIndex = null
+ }
+ )
+ }
+
+ // SettingsDialog removed
+
+ BackHandler(enabled = selectedItemIndex != null) {
+ selectedItemIndex = null
+ }
+
+
}
+
+
+@Preview(showBackground = true)
@Composable
-private fun MakeCall(
- uriHandler: UriHandler,
- conversation: Conversation,
- isTranslateEnabled: Boolean
-) {
- Row(modifier = Modifier
- .padding(8.dp)
- .clickable {
- uriHandler.openUri(conversation.navigationURI.toString())
- }, horizontalArrangement = Arrangement.spacedBy(8.dp)
- ) {
- Box(
- modifier = Modifier
- .background(
- shape = CircleShape,
- color = MaterialTheme.colorScheme.secondaryContainer
+fun ChatScreenContentPreview() {
+ AssistantTheme {
+ ChatScreenContent(
+ isCustomUI = false,
+ isCustomUIHalfPage = false,
+ groupList = listOf(
+ Group(1L, "Recent Conversation 1"),
+ Group(2L, "Recent Conversation 2")
+ ),
+ chatList = listOf(
+ Conversation(
+ text = "Hi! How can I configure my API keys?",
+ isMe = true
),
- ) {
- Icon(
- modifier = Modifier.padding(8.dp),
- imageVector = Icons.Default.Call,
- contentDescription = "Call",
- )
- }
-
- MarkdownText(modifier = Modifier
- .align(alignment = Alignment.CenterVertically)
- .clickable {
- uriHandler.openUri(conversation.navigationURI.toString())
- }, markdown = if(isTranslateEnabled) conversation.translatedText else conversation.englishText
+ Conversation(
+ text = "You can configure them by opening settings in the side drawer menu.",
+ isMe = false
+ )
+ ),
+ isSpeaking = false,
+ isListening = false,
+ isHandsFree = true,
+ question = "",
+ isTranslationEnabled = false,
+ isDeletingChat = false,
+ onGroupClick = {},
+ onSettingsClick = {},
+ onNewChatClick = {},
+ onCopyClick = {},
+ onDeleteClick = {},
+ onClearChatClick = {},
+ onMenuClick = {},
+ onDragEnd = {},
+ onQuestionChange = {},
+ onStopSpeaking = {},
+ onStartListening = {},
+ onProcessQuestion = { _, _, _ -> },
+ onBackPressed = {},
+ onDeleteGroupClick = {}
)
}
}
-@Composable
-private fun ShowWeather(
- uriHandler: UriHandler,
- conversation: Conversation,
- isTranslateEnabled: Boolean
-){
- MarkdownText(modifier = Modifier
- .padding(16.dp)
- .clickable { uriHandler.openUri(conversation.navigationURI.toString()) },
- markdown = if(isTranslateEnabled) conversation.translatedText else conversation.englishText
- )
-}
-
-@Composable
-private fun OtherCard(
- conversation: Conversation,
- onLongClick: (Int) -> Unit,
- index: Int,
- isTranslateEnabled: Boolean
-) {
- MarkdownText(
- markdown = if (isTranslateEnabled && conversation.translatedText.isNotBlank())
- conversation.translatedText
- else
- conversation.englishText,
- modifier = Modifier
- .padding(16.dp, 8.dp)
- .pointerInput(Unit) {
- detectTapGestures(
- onLongPress = {
- onLongClick(index)
- }
- )
- }
- )
-}
-
-@Composable
-fun LoadingDots() {
- val infiniteTransition = rememberInfiniteTransition()
-
- // Animating each dot separately
- val dot1Offset by infiniteTransition.animateFloat(
- initialValue = 0f,
- targetValue = -10f,
- animationSpec = infiniteRepeatable(
- animation = tween(durationMillis = 400, easing = LinearEasing),
- repeatMode = RepeatMode.Reverse,
- initialStartOffset = StartOffset(0) // First dot starts immediately
- ), label = ""
- )
-
- val dot2Offset by infiniteTransition.animateFloat(
- initialValue = 0f,
- targetValue = -10f,
- animationSpec = infiniteRepeatable(
- animation = tween(durationMillis = 400, easing = LinearEasing),
- repeatMode = RepeatMode.Reverse,
- initialStartOffset = StartOffset(200) // Second dot starts with a delay
- ), label = ""
- )
-
- val dot3Offset by infiniteTransition.animateFloat(
- initialValue = 0f,
- targetValue = -10f,
- animationSpec = infiniteRepeatable(
- animation = tween(durationMillis = 400, easing = LinearEasing),
- repeatMode = RepeatMode.Reverse,
- initialStartOffset = StartOffset(400) // Third dot starts with a larger delay
- ), label = ""
- )
-
- // Unicode character for big dot (⏺ or ● for larger circles)
- val dotChar = "\u25CF"
-
- Row(modifier = Modifier.padding(16.dp)) {
- // First dot
- Text(
- text = dotChar,
- modifier = Modifier.offset(y = dot1Offset.dp),
- style = MaterialTheme.typography.bodyLarge
- )
- Spacer(modifier = Modifier.width(4.dp))
-
- // Second dot
- Text(
- text = dotChar,
- modifier = Modifier.offset(y = dot2Offset.dp),
- style = MaterialTheme.typography.bodyLarge
- )
- Spacer(modifier = Modifier.width(4.dp))
-
- // Third dot
- Text(
- text = dotChar,
- modifier = Modifier.offset(y = dot3Offset.dp),
- style = MaterialTheme.typography.bodyLarge
- )
- }
-}
-
-@Composable
-fun MarkdownText(
- modifier: Modifier = Modifier,
- markdown: String
-) {
- val annotatedText = remember(markdown) {
- parseMarkdownToAnnotatedString(markdown)
- }
- Text(
- text = annotatedText,
- modifier = modifier
- )
-}
-
-fun parseMarkdownToAnnotatedString(markdown: String): AnnotatedString {
- val parser = Parser.builder().build()
- val document = parser.parse(markdown)
- val builder = AnnotatedString.Builder()
- processNodes(document, builder)
- return builder.toAnnotatedString()
-}
-
-/**
- * Recursively processes nodes. We only append a newline if there's a *next sibling*,
- * preventing extra blank lines at the very end.
- */
-private fun processNodes(node: Node, builder: AnnotatedString.Builder) {
- var child = node.firstChild
- while (child != null) {
- val nextSibling = child.next
- when (child) {
- is CText -> {
- builder.append(child.literal)
- }
- is Emphasis -> {
- val start = builder.length
- processNodes(child, builder)
- val end = builder.length
- builder.addStyle(SpanStyle(fontStyle = FontStyle.Italic), start, end)
- }
- is StrongEmphasis -> {
- val start = builder.length
- processNodes(child, builder)
- val end = builder.length
- builder.addStyle(SpanStyle(fontWeight = FontWeight.Bold), start, end)
- }
- is Code -> {
- // Inline code (single backticks)
- val start = builder.length
- builder.append(child.literal)
- val end = builder.length
- // Monospace but keep background transparent for inline code
- builder.addStyle(
- SpanStyle(fontFamily = FontFamily.Monospace),
- start,
- end
- )
- }
- is Link -> {
- val start = builder.length
- processNodes(child, builder)
- val end = builder.length
- // Annotate the text with the URL (useful if you want to make it clickable later)
- builder.addStringAnnotation(
- tag = "URL",
- annotation = child.destination,
- start = start,
- end = end
- )
- // Style the link text
- builder.addStyle(
- SpanStyle(color = Color.Blue, textDecoration = TextDecoration.Underline),
- start,
- end
- )
- }
- is Paragraph -> {
- // Process paragraph content
- processNodes(child, builder)
- // Only append a newline if there's another sibling after this paragraph
- if (nextSibling != null) {
- builder.append("\n")
- }
- }
- is Heading -> {
- // Insert a newline before headings (for spacing), unless it's the first node
- if (builder.isNotEmpty() && !builder.toString().endsWith("\n")) {
- builder.append("\n")
- }
- val start = builder.length
- processNodes(child, builder)
- val end = builder.length
- // Map heading level to style
- val headingStyle = when (child.level) {
- 1 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 28.sp)
- 2 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 24.sp)
- 3 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 20.sp)
- 4 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 18.sp)
- 5 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 16.sp)
- else -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 14.sp)
- }
- builder.addStyle(headingStyle, start, end)
- // Only append a newline if there's another sibling
- if (nextSibling != null) {
- builder.append("\n")
- }
- }
- is BlockQuote -> {
- // Optional: style block quotes in italics or different color
- val start = builder.length
- processNodes(child, builder)
- val end = builder.length
- builder.addStyle(
- SpanStyle(color = Color.Gray, fontStyle = FontStyle.Italic),
- start,
- end
- )
- if (nextSibling != null) {
- builder.append("\n")
- }
- }
- is BulletList -> {
- var listItem = child.firstChild
- while (listItem != null) {
- if (listItem is ListItem) {
- // Prefix with a bullet
- builder.append("• ")
- processNodes(listItem, builder)
- builder.append("\n")
- }
- listItem = listItem.next
- }
- if (nextSibling != null) {
- builder.append("\n")
- }
- }
- is OrderedList -> {
- var index = child.startNumber
- var listItem = child.firstChild
- while (listItem != null) {
- if (listItem is ListItem) {
- builder.append("$index. ")
- processNodes(listItem, builder)
- builder.append("\n")
- index++
- }
- listItem = listItem.next
- }
- if (nextSibling != null) {
- builder.append("\n")
- }
- }
- is FencedCodeBlock -> {
- // Code block with triple backticks
- val start = builder.length
- // Insert the code exactly as-is
- builder.append(child.literal)
- val end = builder.length
- // Always dark background, white text
- builder.addStyle(
- SpanStyle(
- fontFamily = FontFamily.Monospace
- ),
- start,
- end
- )
- // Only append a newline if there's another sibling
- if (nextSibling != null) {
- builder.append("\n")
- }
- }
- is IndentedCodeBlock -> {
- val start = builder.length
- builder.append(child.literal)
- val end = builder.length
- builder.addStyle(
- SpanStyle(
- fontFamily = FontFamily.Monospace
- ),
- start,
- end
- )
- if (nextSibling != null) {
- builder.append("\n")
- }
- }
- is ThematicBreak -> {
- builder.append("\n----------------\n")
- if (nextSibling != null) {
- builder.append("\n")
- }
- }
- is SoftLineBreak -> {
- // Usually rendered as a space in Markdown
- builder.append(" ")
- }
- is HardLineBreak -> {
- builder.append("\n")
- }
- else -> {
- processNodes(child, builder)
- }
- }
- child = nextSibling
- }
-}
-
-/** Quick helper to check if the builder already has text. */
-private fun AnnotatedString.Builder.isNotEmpty(): Boolean {
- return this.length > 0
-}
\ No newline at end of file
diff --git a/app/src/main/java/com/app/assistant/ui/screen/ChatScreenComponents.kt b/app/src/main/java/com/app/assistant/ui/screen/ChatScreenComponents.kt
new file mode 100644
index 0000000..ad8ed7e
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/ChatScreenComponents.kt
@@ -0,0 +1,387 @@
+package com.app.assistant.ui.screen
+
+import androidx.compose.foundation.background
+import androidx.compose.foundation.clickable
+import androidx.compose.foundation.gestures.detectDragGestures
+import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.Column
+import androidx.compose.foundation.layout.Row
+import androidx.compose.foundation.layout.fillMaxSize
+import androidx.compose.foundation.layout.fillMaxWidth
+import androidx.compose.foundation.layout.height
+import androidx.compose.foundation.layout.imePadding
+import androidx.compose.foundation.layout.padding
+import androidx.compose.foundation.layout.width
+import androidx.compose.foundation.lazy.LazyColumn
+import androidx.compose.foundation.lazy.items
+import androidx.compose.foundation.rememberScrollState
+import androidx.compose.foundation.shape.RoundedCornerShape
+import androidx.compose.foundation.verticalScroll
+import androidx.compose.material3.AlertDialog
+import androidx.compose.material3.ExperimentalMaterial3Api
+import androidx.compose.material3.HorizontalDivider
+import androidx.compose.material3.Icon
+import androidx.compose.material3.IconButton
+import androidx.compose.foundation.ExperimentalFoundationApi
+import androidx.compose.foundation.combinedClickable
+import androidx.compose.material3.DropdownMenu
+import androidx.compose.material3.DropdownMenuItem
+import androidx.compose.material3.LinearProgressIndicator
+import androidx.compose.ui.draw.clip
+import androidx.compose.material3.MaterialTheme
+import androidx.compose.material3.ModalBottomSheet
+import androidx.compose.material3.ModalDrawerSheet
+import androidx.compose.material3.NavigationDrawerItem
+import androidx.compose.material3.SheetState
+import androidx.compose.material3.Switch
+import androidx.compose.material3.Text
+import androidx.compose.material3.TextButton
+import androidx.compose.material3.TextField
+import androidx.compose.material3.TextFieldDefaults
+import androidx.compose.material3.TopAppBar
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.mutableStateOf
+import androidx.compose.runtime.remember
+import androidx.compose.runtime.setValue
+import androidx.compose.ui.Alignment
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.input.pointer.pointerInput
+import androidx.compose.ui.res.painterResource
+import androidx.compose.ui.res.stringResource
+import androidx.compose.ui.unit.dp
+import com.app.assistant.R
+import com.app.assistant.model.Group
+import androidx.compose.ui.text.style.TextOverflow
+import androidx.compose.ui.tooling.preview.Preview
+import com.app.assistant.ui.theme.AssistantTheme
+
+@OptIn(ExperimentalFoundationApi::class)
+@Composable
+fun ChatDrawerItem(
+ group: Group,
+ isDeletingChat: Boolean,
+ onGroupClick: (Long) -> Unit,
+ onDeleteGroupClick: (Long) -> Unit,
+) {
+ var showMenu by remember { mutableStateOf(false) }
+ var showConfirmDialog by remember { mutableStateOf(false) }
+
+ Box(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 4.dp)
+ .clip(RoundedCornerShape(28.dp))
+ .combinedClickable(
+ enabled = !isDeletingChat,
+ onClick = { onGroupClick(group.groupId) },
+ onLongClick = { showMenu = true }
+ )
+ .padding(horizontal = 16.dp, vertical = 14.dp)
+ ) {
+ Text(
+ text = group.title,
+ maxLines = 1,
+ overflow = TextOverflow.Ellipsis,
+ color = MaterialTheme.colorScheme.onSurfaceVariant
+ )
+
+ DropdownMenu(
+ expanded = showMenu,
+ onDismissRequest = { showMenu = false }
+ ) {
+ DropdownMenuItem(
+ text = { Text(stringResource(id = R.string.delete)) },
+ onClick = {
+ showMenu = false
+ showConfirmDialog = true
+ }
+ )
+ }
+ }
+
+ if (showConfirmDialog) {
+ AlertDialog(
+ onDismissRequest = { showConfirmDialog = false },
+ text = { Text(stringResource(id = R.string.clear_chat_history_confirm)) },
+ confirmButton = {
+ TextButton(
+ onClick = {
+ showConfirmDialog = false
+ onDeleteGroupClick(group.groupId)
+ }
+ ) {
+ Text(stringResource(id = R.string.yes))
+ }
+ },
+ dismissButton = {
+ TextButton(onClick = { showConfirmDialog = false }) {
+ Text(stringResource(id = R.string.no))
+ }
+ }
+ )
+ }
+}
+
+@OptIn(ExperimentalFoundationApi::class)
+@Composable
+fun ChatDrawerContent(
+ groupList: List,
+ isDeletingChat: Boolean,
+ onGroupClick: (Long) -> Unit,
+ onSettingsClick: () -> Unit,
+ onNewChatClick: () -> Unit,
+ onDeleteGroupClick: (Long) -> Unit,
+) {
+ ModalDrawerSheet {
+ Row(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(16.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ Text(stringResource(id = R.string.chats))
+ }
+
+ if (isDeletingChat) {
+ LinearProgressIndicator(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(horizontal = 16.dp, vertical = 8.dp)
+ )
+ }
+
+ Column(
+ modifier =
+ Modifier
+ .padding(horizontal = 8.dp)
+ .weight(1f)
+ .verticalScroll(rememberScrollState()),
+ ) {
+ groupList.forEach { group ->
+ ChatDrawerItem(
+ group = group,
+ isDeletingChat = isDeletingChat,
+ onGroupClick = onGroupClick,
+ onDeleteGroupClick = onDeleteGroupClick
+ )
+ }
+ }
+ HorizontalDivider(modifier = Modifier.padding(vertical = 8.dp))
+ NavigationDrawerItem(
+ label = { Text(text = stringResource(id = R.string.settings_title)) },
+ icon = {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_settings),
+ contentDescription = stringResource(id = R.string.settings_title),
+ )
+ },
+ selected = false,
+ onClick = {
+ if (!isDeletingChat) {
+ onSettingsClick()
+ }
+ },
+ )
+ NavigationDrawerItem(
+ label = { Text(text = stringResource(id = R.string.start_new_chat)) },
+ icon = {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_add_new_chat),
+ contentDescription = stringResource(id = R.string.start_new_chat),
+ )
+ },
+ selected = false,
+ onClick = {
+ if (!isDeletingChat) {
+ onNewChatClick()
+ }
+ },
+ )
+ }
+}
+
+@OptIn(ExperimentalMaterial3Api::class)
+@Composable
+fun ChatTopAppBar(
+ showCopyIcon: Boolean,
+ onCopyClick: () -> Unit,
+ onDeleteClick: () -> Unit,
+ chatListNotEmpty: Boolean,
+ onClearChatClick: () -> Unit,
+ onMenuClick: () -> Unit,
+) {
+ TopAppBar(
+ navigationIcon = {
+ IconButton(onClick = onMenuClick) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_menu),
+ contentDescription = stringResource(id = R.string.menu),
+ )
+ }
+ },
+ title = { Text("") },
+ actions = {
+ if (showCopyIcon) {
+ IconButton(onClick = onCopyClick) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_content_copy),
+ contentDescription = stringResource(id = R.string.copy),
+ )
+ }
+ IconButton(onClick = onDeleteClick) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_delete),
+ contentDescription = stringResource(id = R.string.delete),
+ )
+ }
+ }
+ if (chatListNotEmpty) {
+ IconButton(onClick = onClearChatClick) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_restart),
+ contentDescription = stringResource(id = R.string.restart_chat),
+ )
+ }
+ }
+ },
+ )
+}
+
+@Composable
+fun CustomUiDragHandle(
+ onDragEnd: () -> Unit,
+) {
+ Box(
+ modifier =
+ Modifier
+ .fillMaxWidth()
+ .height(24.dp)
+ .then(
+ Modifier.pointerInput(Unit) {
+ detectDragGestures(
+ onDragStart = { /* Optional: trigger on drag start */ },
+ onDragEnd = onDragEnd,
+ onDrag = { _, _ ->
+ // track dragAmount if needed
+ },
+ )
+ },
+ )
+ .clickable { onDragEnd() },
+ contentAlignment = Alignment.Center,
+ ) {
+ Box(
+ modifier =
+ Modifier
+ .width(36.dp)
+ .height(4.dp)
+ .background(
+ color = Color.Gray,
+ shape = RoundedCornerShape(2.dp),
+ ),
+ )
+ }
+}
+
+// LanguageSelectionBottomSheet removed (moved to Settings)
+
+@Composable
+fun ClearChatDialog(
+ onConfirm: () -> Unit,
+ onDismiss: () -> Unit,
+) {
+ AlertDialog(
+ onDismissRequest = onDismiss,
+ text = { Text(stringResource(id = R.string.clear_chat_history_confirm)) },
+ confirmButton = {
+ TextButton(onClick = onConfirm) {
+ Text(stringResource(id = R.string.yes))
+ }
+ },
+ dismissButton = {
+ TextButton(onClick = onDismiss) {
+ Text(stringResource(id = R.string.no))
+ }
+ },
+ )
+}
+
+@Composable
+fun DeleteMessageDialog(
+ onConfirm: () -> Unit,
+ onDismiss: () -> Unit,
+) {
+ AlertDialog(
+ onDismissRequest = onDismiss,
+ text = { Text(stringResource(id = R.string.delete_message_confirm)) },
+ confirmButton = {
+ TextButton(onClick = onConfirm) {
+ Text(stringResource(id = R.string.yes))
+ }
+ },
+ dismissButton = {
+ TextButton(onClick = onDismiss) {
+ Text(stringResource(id = R.string.no))
+ }
+ },
+ )
+}
+
+@Preview(showBackground = true)
+@Composable
+fun ChatDrawerContentPreview() {
+ AssistantTheme {
+ ChatDrawerContent(
+ groupList = listOf(
+ Group(1L, "Recent Conversation 1"),
+ Group(2L, "Recent Conversation 2")
+ ),
+ isDeletingChat = false,
+ onGroupClick = {},
+ onSettingsClick = {},
+ onNewChatClick = {},
+ onDeleteGroupClick = {}
+ )
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun ChatTopAppBarPreview() {
+ AssistantTheme {
+ ChatTopAppBar(
+ showCopyIcon = true,
+ onCopyClick = {},
+ onDeleteClick = {},
+ chatListNotEmpty = true,
+ onClearChatClick = {},
+ onMenuClick = {}
+ )
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun CustomUiDragHandlePreview() {
+ AssistantTheme {
+ CustomUiDragHandle(onDragEnd = {})
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun ClearChatDialogPreview() {
+ AssistantTheme {
+ ClearChatDialog(onConfirm = {}, onDismiss = {})
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun DeleteMessageDialogPreview() {
+ AssistantTheme {
+ DeleteMessageDialog(onConfirm = {}, onDismiss = {})
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/screen/ConversationItem.kt b/app/src/main/java/com/app/assistant/ui/screen/ConversationItem.kt
new file mode 100644
index 0000000..bbd28a9
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/ConversationItem.kt
@@ -0,0 +1,895 @@
+package com.app.assistant.ui.screen
+
+import androidx.compose.foundation.background
+import androidx.compose.foundation.clickable
+import androidx.compose.foundation.gestures.detectTapGestures
+import androidx.compose.foundation.layout.Arrangement
+import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.Column
+import androidx.compose.foundation.layout.Row
+import androidx.compose.foundation.layout.Spacer
+import androidx.compose.foundation.layout.fillMaxWidth
+import androidx.compose.foundation.layout.padding
+import androidx.compose.foundation.layout.width
+import androidx.compose.foundation.layout.widthIn
+import androidx.compose.foundation.layout.wrapContentWidth
+import androidx.compose.foundation.shape.CircleShape
+import androidx.compose.foundation.shape.RoundedCornerShape
+import androidx.compose.material.icons.Icons
+import androidx.compose.material.icons.filled.Call
+import androidx.compose.material.icons.filled.LocationOn
+import androidx.compose.material3.Card
+import androidx.compose.material3.CardDefaults
+import androidx.compose.material3.Icon
+import androidx.compose.material3.MaterialTheme
+import androidx.compose.material3.Text
+import androidx.compose.runtime.Composable
+import androidx.compose.ui.Alignment
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.input.pointer.pointerInput
+import androidx.compose.ui.platform.LocalConfiguration
+import androidx.compose.ui.platform.LocalUriHandler
+import androidx.compose.ui.platform.UriHandler
+import androidx.compose.ui.text.style.TextAlign
+import androidx.compose.ui.text.style.TextOverflow
+import androidx.compose.ui.unit.Dp
+import androidx.compose.ui.unit.dp
+import androidx.compose.ui.res.stringResource
+import coil3.compose.AsyncImage
+import com.app.assistant.R
+import com.app.assistant.model.Conversation
+import com.app.assistant.model.Attachment
+import com.app.assistant.util.Category
+import androidx.compose.ui.tooling.preview.Preview
+import androidx.compose.ui.unit.sp
+import com.app.assistant.ui.theme.AssistantTheme
+import androidx.compose.animation.AnimatedVisibility
+import androidx.compose.animation.core.animateFloatAsState
+import androidx.compose.animation.core.tween
+import androidx.compose.foundation.layout.height
+import androidx.compose.foundation.layout.size
+import androidx.compose.material.icons.filled.KeyboardArrowDown
+import androidx.compose.material.icons.filled.KeyboardArrowUp
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.setValue
+import androidx.compose.runtime.mutableStateOf
+import androidx.compose.runtime.remember
+import androidx.compose.ui.text.font.FontWeight
+import androidx.compose.foundation.Image
+import androidx.compose.ui.graphics.asImageBitmap
+import androidx.compose.ui.layout.ContentScale
+import androidx.compose.runtime.rememberCoroutineScope
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import kotlinx.coroutines.launch
+import kotlinx.coroutines.delay
+import androidx.compose.ui.platform.LocalContext
+import androidx.compose.material3.CircularProgressIndicator
+import androidx.compose.runtime.LaunchedEffect
+import androidx.compose.ui.draw.clip
+import androidx.compose.material3.IconButton
+import androidx.compose.ui.res.painterResource
+
+@Composable
+@Suppress("ktlint:standard:function-naming")
+fun ConversationItem(
+ conversation: Conversation,
+ index: Int,
+ isSelected: Boolean,
+ onLongClick: (Int) -> Unit,
+ isTranslateEnabled: Boolean,
+ backgroundColor: Color =
+ if (conversation.isMe) {
+ MaterialTheme.colorScheme.secondary
+ } else {
+ MaterialTheme.colorScheme.surfaceContainerHighest
+ },
+ cornerRadius: Dp = 20.dp,
+) {
+ val uriHandler = LocalUriHandler.current
+ Column(
+ modifier = Modifier.fillMaxWidth(),
+ ) {
+ Row(
+ modifier =
+ Modifier
+ .fillMaxWidth()
+ .padding(horizontal = 8.dp),
+ horizontalArrangement = if (conversation.isMe) Arrangement.Start else Arrangement.End,
+ ) {
+ if (conversation.isMe) {
+ Spacer(modifier = Modifier.weight(1f))
+ }
+ Card(
+ Modifier
+ .wrapContentWidth()
+ .widthIn(max = LocalConfiguration.current.screenWidthDp.dp * 0.85f),
+ shape = RoundedCornerShape(cornerRadius),
+ colors =
+ CardDefaults.cardColors(
+ containerColor =
+ if (isSelected) {
+ MaterialTheme.colorScheme.secondaryContainer
+ } else {
+ backgroundColor
+ },
+ ),
+ ) {
+ if (conversation.isLoading) {
+ LoadingDots() // Show loading animation when isLoading is true
+ } else {
+ if (conversation.isMe) {
+ OtherCard(conversation, onLongClick, index, cornerRadius)
+ } else {
+ when (conversation.category) {
+ Category.CALL.name -> {
+ MakeCall(uriHandler, conversation)
+ }
+
+ Category.OTHER.name -> {
+ OtherCard(conversation, onLongClick, index, cornerRadius)
+ }
+
+ Category.SETTINGS.name -> {
+ // TODO
+ OtherCard(conversation, onLongClick, index, cornerRadius)
+ }
+
+ Category.SONGS.name -> {
+ PlaySong(uriHandler, conversation)
+ }
+
+ Category.NAVIGATION.name -> {
+ StartNavigation(uriHandler, conversation)
+ }
+
+ Category.WEATHER.name -> {
+ ShowWeather(uriHandler, conversation)
+ }
+
+ Category.REMINDER.name -> {
+ OtherCard(conversation, onLongClick, index, cornerRadius)
+ }
+
+ Category.ALARM.name -> {
+ OtherCard(conversation, onLongClick, index, cornerRadius)
+ }
+
+ else -> {
+ OtherCard(conversation, onLongClick, index, cornerRadius)
+ }
+ }
+ }
+ }
+ }
+ if (!conversation.isMe) {
+ Spacer(modifier = Modifier.weight(1f))
+ }
+ }
+
+ if (conversation.isMe) {
+ Text(
+ text = conversation.category,
+ modifier =
+ Modifier
+ .padding(0.dp, 0.dp, 20.dp, 0.dp)
+ .fillMaxWidth(),
+ maxLines = 1,
+ overflow = TextOverflow.Ellipsis,
+ color = MaterialTheme.colorScheme.secondary,
+ textAlign = TextAlign.End,
+ fontSize = 10.sp
+ )
+ }
+ }
+}
+
+@Composable
+private fun StartNavigation(
+ uriHandler: UriHandler,
+ conversation: Conversation,
+) {
+ Row(
+ modifier =
+ Modifier
+ .padding(8.dp)
+ .clickable {
+ uriHandler.openUri(conversation.navigationURI.toString())
+ },
+ horizontalArrangement = Arrangement.spacedBy(8.dp),
+ ) {
+ Box(
+ modifier =
+ Modifier
+ .background(
+ shape = CircleShape,
+ color = MaterialTheme.colorScheme.secondaryContainer,
+ ),
+ ) {
+ Icon(
+ modifier = Modifier.padding(8.dp),
+ imageVector = Icons.Default.LocationOn,
+ contentDescription = stringResource(id = R.string.navigation_desc),
+ )
+ }
+
+ MarkdownText(
+ modifier =
+ Modifier
+ .align(alignment = Alignment.CenterVertically),
+ markdown = conversation.text,
+ )
+ }
+}
+
+@Composable
+private fun PlaySong(
+ uriHandler: UriHandler,
+ conversation: Conversation,
+) {
+ AsyncImage(
+ modifier =
+ Modifier
+ .clickable {
+ uriHandler.openUri(conversation.navigationURI.toString())
+ },
+ model = conversation.contentURL,
+ contentDescription = conversation.text,
+ )
+}
+
+@Composable
+private fun MakeCall(
+ uriHandler: UriHandler,
+ conversation: Conversation,
+) {
+ Row(
+ modifier =
+ Modifier
+ .padding(8.dp)
+ .clickable {
+ uriHandler.openUri(conversation.navigationURI.toString())
+ },
+ horizontalArrangement = Arrangement.spacedBy(8.dp),
+ ) {
+ Box(
+ modifier =
+ Modifier
+ .background(
+ shape = CircleShape,
+ color = MaterialTheme.colorScheme.secondaryContainer,
+ ),
+ ) {
+ Icon(
+ modifier = Modifier.padding(8.dp),
+ imageVector = Icons.Default.Call,
+ contentDescription = stringResource(id = R.string.call_desc),
+ )
+ }
+
+ MarkdownText(
+ modifier =
+ Modifier
+ .align(alignment = Alignment.CenterVertically),
+ markdown = conversation.text,
+ )
+ }
+}
+
+@Composable
+private fun ShowWeather(
+ uriHandler: UriHandler,
+ conversation: Conversation,
+) {
+ MarkdownText(
+ modifier =
+ Modifier
+ .padding(16.dp)
+ .clickable { uriHandler.openUri(conversation.navigationURI.toString()) },
+ markdown = conversation.text,
+ )
+}
+
+@Composable
+private fun OtherCard(
+ conversation: Conversation,
+ onLongClick: (Int) -> Unit,
+ index: Int,
+ cornerRadius: Dp = 20.dp,
+) {
+ val thinkingText = conversation.getThinkingProcess()
+ val answerText = conversation.getActualAnswer()
+ val hasText = answerText.isNotEmpty() || conversation.isStreaming || thinkingText != null
+
+ Column(
+ modifier = Modifier
+ .pointerInput(Unit) {
+ detectTapGestures(
+ onLongPress = {
+ onLongClick(index)
+ },
+ )
+ }
+ ) {
+ if (conversation.attachments.isNotEmpty()) {
+ ConversationAttachments(
+ attachments = conversation.attachments,
+ hasTextBelow = hasText,
+ cornerRadius = cornerRadius,
+ modifier = Modifier
+ )
+ }
+
+ if (thinkingText != null || answerText.isNotEmpty() || conversation.isStreaming) {
+ Column(
+ modifier = Modifier.padding(
+ start = 16.dp,
+ end = 16.dp,
+ top = if (conversation.attachments.isNotEmpty()) 8.dp else 8.dp,
+ bottom = 8.dp
+ )
+ ) {
+ if (thinkingText != null) {
+ var isExpanded by remember { mutableStateOf(true) }
+
+ Card(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(bottom = 8.dp),
+ shape = RoundedCornerShape(12.dp),
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.5f)
+ )
+ ) {
+ Column(modifier = Modifier.padding(12.dp)) {
+ Row(
+ modifier = Modifier
+ .fillMaxWidth()
+ .clickable { isExpanded = !isExpanded },
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ Text(
+ text = "🧠",
+ fontSize = 16.sp,
+ modifier = Modifier.padding(end = 8.dp)
+ )
+ Text(
+ text = "Thinking Process",
+ style = MaterialTheme.typography.labelLarge.copy(
+ fontWeight = FontWeight.Bold,
+ color = MaterialTheme.colorScheme.primary
+ ),
+ modifier = Modifier.weight(1f)
+ )
+ Icon(
+ imageVector = if (isExpanded) {
+ Icons.Default.KeyboardArrowUp
+ } else {
+ Icons.Default.KeyboardArrowDown
+ },
+ contentDescription = if (isExpanded) "Collapse" else "Expand",
+ tint = MaterialTheme.colorScheme.primary,
+ modifier = Modifier.size(20.dp)
+ )
+ }
+
+ AnimatedVisibility(visible = isExpanded) {
+ Column {
+ Spacer(modifier = Modifier.height(8.dp))
+ Text(
+ text = thinkingText,
+ style = MaterialTheme.typography.bodyMedium.copy(
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.8f)
+ ),
+ modifier = Modifier.fillMaxWidth()
+ )
+ }
+ }
+ }
+ }
+ }
+
+ if (answerText.isNotEmpty() || conversation.isStreaming) {
+ val displayAnswer = if (conversation.isStreaming) {
+ "$answerText ▮"
+ } else {
+ answerText
+ }
+
+ MarkdownText(
+ markdown = displayAnswer,
+ modifier = Modifier
+ )
+ }
+ }
+ }
+ }
+}
+
+@Composable
+fun ConversationAttachments(
+ attachments: List,
+ hasTextBelow: Boolean,
+ cornerRadius: Dp,
+ modifier: Modifier = Modifier
+) {
+ Column(
+ modifier = modifier.fillMaxWidth(),
+ verticalArrangement = Arrangement.spacedBy(8.dp)
+ ) {
+ attachments.forEachIndexed { index, attachment ->
+ val isFirst = index == 0
+ val isLast = index == attachments.size - 1
+ when {
+ attachment.mimeType.startsWith("image/") -> {
+ ImageAttachmentBubble(
+ attachment = attachment,
+ isFirst = isFirst,
+ isLast = isLast,
+ hasTextBelow = hasTextBelow,
+ cornerRadius = cornerRadius
+ )
+ }
+ attachment.mimeType.startsWith("audio/") -> {
+ Box(modifier = Modifier.padding(horizontal = 16.dp, vertical = 4.dp)) {
+ AudioAttachmentBubble(attachment)
+ }
+ }
+ attachment.mimeType.startsWith("video/") -> {
+ Box(modifier = Modifier.padding(horizontal = 16.dp, vertical = 4.dp)) {
+ VideoAttachmentBubble(attachment)
+ }
+ }
+ else -> {
+ Box(modifier = Modifier.padding(horizontal = 16.dp, vertical = 4.dp)) {
+ DocumentAttachmentBubble(attachment)
+ }
+ }
+ }
+ }
+ }
+}
+
+@Composable
+fun ImageAttachmentBubble(
+ attachment: Attachment,
+ isFirst: Boolean,
+ isLast: Boolean,
+ hasTextBelow: Boolean,
+ cornerRadius: Dp
+) {
+ val context = LocalContext.current
+ val coroutineScope = rememberCoroutineScope()
+ var bitmap by remember { mutableStateOf(null) }
+ var isLoading by remember { mutableStateOf(true) }
+ var imageLoaded by remember { mutableStateOf(false) }
+
+ LaunchedEffect(attachment) {
+ isLoading = true
+ imageLoaded = false
+ withContext(Dispatchers.IO) {
+ try {
+ val file = java.io.File(attachment.filePath)
+ if (file.exists()) {
+ val fileBytes = file.readBytes()
+ val decrypted = com.app.assistant.db.EncryptionUtil.decryptFile(fileBytes, attachment.iv)
+ val decoded = android.graphics.BitmapFactory.decodeByteArray(decrypted, 0, decrypted.size)
+ bitmap = decoded
+ if (decoded != null) {
+ imageLoaded = true
+ }
+ }
+ } catch (e: Exception) {
+ e.printStackTrace()
+ } finally {
+ isLoading = false
+ }
+ }
+ }
+
+ val topRounding = if (isFirst) cornerRadius else 0.dp
+ val bottomRounding = if (isLast && !hasTextBelow) cornerRadius else 0.dp
+ val imageShape = RoundedCornerShape(
+ topStart = topRounding,
+ topEnd = topRounding,
+ bottomStart = bottomRounding,
+ bottomEnd = bottomRounding
+ )
+
+ val imageAlpha by animateFloatAsState(
+ targetValue = if (imageLoaded) 1f else 0f,
+ animationSpec = tween(durationMillis = 400),
+ label = "imageFadeIn"
+ )
+
+ Box(
+ modifier = Modifier
+ .fillMaxWidth()
+ .clip(imageShape)
+ .background(MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.3f))
+ .clickable {
+ coroutineScope.launch(Dispatchers.IO) {
+ try {
+ val fileBytes = java.io.File(attachment.filePath).readBytes()
+ val decrypted = com.app.assistant.db.EncryptionUtil.decryptFile(fileBytes, attachment.iv)
+ val ext = attachment.fileName.substringAfterLast('.', "jpg")
+ val temp = java.io.File(context.cacheDir, "view_${attachment.id}.$ext")
+ temp.writeBytes(decrypted)
+
+ val uri = androidx.core.content.FileProvider.getUriForFile(
+ context,
+ "com.app.assistant.fileprovider",
+ temp
+ )
+ val intent = android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
+ setDataAndType(uri, "image/*")
+ addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
+ }
+ context.startActivity(intent)
+ } catch (e: Exception) {
+ withContext(Dispatchers.Main) {
+ android.widget.Toast.makeText(context, "Cannot open image: ${e.message}", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ }
+ }
+ },
+ contentAlignment = Alignment.Center
+ ) {
+ if (isLoading) {
+ Box(
+ modifier = Modifier
+ .fillMaxWidth()
+ .height(240.dp),
+ contentAlignment = Alignment.Center
+ ) {
+ CircularProgressIndicator(
+ modifier = Modifier.size(36.dp),
+ color = MaterialTheme.colorScheme.primary,
+ strokeWidth = 3.dp
+ )
+ }
+ } else if (bitmap != null) {
+ Image(
+ bitmap = bitmap!!.asImageBitmap(),
+ contentDescription = null,
+ modifier = Modifier
+ .fillMaxWidth()
+ .height(240.dp),
+ contentScale = ContentScale.Crop,
+ alpha = imageAlpha
+ )
+ } else {
+ Box(
+ modifier = Modifier
+ .fillMaxWidth()
+ .height(100.dp),
+ contentAlignment = Alignment.Center
+ ) {
+ Text(
+ text = "Failed to load image",
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.error
+ )
+ }
+ }
+ }
+}
+
+@Composable
+fun AudioAttachmentBubble(attachment: Attachment) {
+ val context = LocalContext.current
+ var isPlaying by remember { mutableStateOf(false) }
+ var progress by remember { mutableStateOf(0f) }
+ var duration by remember { mutableStateOf(0) }
+ var currentPosition by remember { mutableStateOf(0) }
+ val mediaPlayer = remember { android.media.MediaPlayer() }
+ var tempFile by remember { mutableStateOf(null) }
+ val coroutineScope = rememberCoroutineScope()
+
+ androidx.compose.runtime.DisposableEffect(Unit) {
+ onDispose {
+ try {
+ if (mediaPlayer.isPlaying) {
+ mediaPlayer.stop()
+ }
+ mediaPlayer.release()
+ } catch (e: Exception) {}
+ tempFile?.delete()
+ }
+ }
+
+ LaunchedEffect(isPlaying) {
+ if (isPlaying) {
+ while (isPlaying) {
+ try {
+ currentPosition = mediaPlayer.currentPosition
+ progress = if (duration > 0) currentPosition.toFloat() / duration else 0f
+ } catch (e: Exception) {}
+ kotlinx.coroutines.delay(200)
+ }
+ }
+ }
+
+ Card(
+ colors = CardDefaults.cardColors(containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.5f)),
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth().padding(vertical = 4.dp)
+ ) {
+ Row(
+ modifier = Modifier.padding(8.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ IconButton(onClick = {
+ if (isPlaying) {
+ mediaPlayer.pause()
+ isPlaying = false
+ } else {
+ if (tempFile == null) {
+ coroutineScope.launch(Dispatchers.IO) {
+ try {
+ val fileBytes = java.io.File(attachment.filePath).readBytes()
+ val decrypted = com.app.assistant.db.EncryptionUtil.decryptFile(fileBytes, attachment.iv)
+ val temp = java.io.File(context.cacheDir, "play_${attachment.id}.mp3")
+ temp.writeBytes(decrypted)
+ tempFile = temp
+
+ withContext(Dispatchers.Main) {
+ mediaPlayer.reset()
+ mediaPlayer.setDataSource(temp.absolutePath)
+ mediaPlayer.prepare()
+ duration = mediaPlayer.duration
+ mediaPlayer.start()
+ isPlaying = true
+ mediaPlayer.setOnCompletionListener {
+ isPlaying = false
+ progress = 0f
+ currentPosition = 0
+ }
+ }
+ } catch (e: Exception) {
+ e.printStackTrace()
+ }
+ }
+ } else {
+ mediaPlayer.start()
+ isPlaying = true
+ }
+ }
+ }) {
+ Icon(
+ painter = painterResource(id = if (isPlaying) R.drawable.ic_stop else R.drawable.ic_mic),
+ contentDescription = if (isPlaying) "Pause" else "Play",
+ tint = MaterialTheme.colorScheme.primary
+ )
+ }
+ Spacer(modifier = Modifier.width(8.dp))
+ Column(modifier = Modifier.weight(1f)) {
+ Text(
+ text = attachment.fileName,
+ style = MaterialTheme.typography.labelMedium,
+ maxLines = 1,
+ overflow = TextOverflow.Ellipsis
+ )
+ Spacer(modifier = Modifier.height(4.dp))
+ androidx.compose.material3.LinearProgressIndicator(
+ progress = { progress },
+ modifier = Modifier.fillMaxWidth().height(4.dp),
+ color = MaterialTheme.colorScheme.primary,
+ trackColor = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.2f)
+ )
+ }
+ }
+ }
+}
+
+@Composable
+fun VideoAttachmentBubble(attachment: Attachment) {
+ val context = LocalContext.current
+ val coroutineScope = rememberCoroutineScope()
+ Card(
+ colors = CardDefaults.cardColors(containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.5f)),
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth().padding(vertical = 4.dp).clickable {
+ coroutineScope.launch(Dispatchers.IO) {
+ try {
+ val fileBytes = java.io.File(attachment.filePath).readBytes()
+ val decrypted = com.app.assistant.db.EncryptionUtil.decryptFile(fileBytes, attachment.iv)
+ val ext = attachment.fileName.substringAfterLast('.', "mp4")
+ val temp = java.io.File(context.cacheDir, "play_${attachment.id}.$ext")
+ temp.writeBytes(decrypted)
+
+ val uri = androidx.core.content.FileProvider.getUriForFile(
+ context,
+ "com.app.assistant.fileprovider",
+ temp
+ )
+ val intent = android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
+ setDataAndType(uri, "video/*")
+ addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
+ }
+ context.startActivity(intent)
+ } catch (e: Exception) {
+ withContext(Dispatchers.Main) {
+ android.widget.Toast.makeText(context, "Cannot play video: ${e.message}", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ }
+ }
+ }
+ ) {
+ Row(
+ modifier = Modifier.padding(12.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_stop),
+ contentDescription = "Play Video",
+ tint = MaterialTheme.colorScheme.primary,
+ modifier = Modifier.size(32.dp)
+ )
+ Spacer(modifier = Modifier.width(12.dp))
+ Column {
+ Text(
+ text = attachment.fileName,
+ style = MaterialTheme.typography.bodyMedium,
+ fontWeight = FontWeight.Bold,
+ maxLines = 1,
+ overflow = TextOverflow.Ellipsis
+ )
+ Text(
+ text = "Tap to play video",
+ style = MaterialTheme.typography.labelSmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.7f)
+ )
+ }
+ }
+ }
+}
+
+@Composable
+fun DocumentAttachmentBubble(attachment: Attachment) {
+ val context = LocalContext.current
+ val coroutineScope = rememberCoroutineScope()
+ Card(
+ colors = CardDefaults.cardColors(containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.5f)),
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth().padding(vertical = 4.dp).clickable {
+ coroutineScope.launch(Dispatchers.IO) {
+ try {
+ val fileBytes = java.io.File(attachment.filePath).readBytes()
+ val decrypted = com.app.assistant.db.EncryptionUtil.decryptFile(fileBytes, attachment.iv)
+ val ext = attachment.fileName.substringAfterLast('.', "pdf")
+ val temp = java.io.File(context.cacheDir, "doc_${attachment.id}.$ext")
+ temp.writeBytes(decrypted)
+
+ val uri = androidx.core.content.FileProvider.getUriForFile(
+ context,
+ "com.app.assistant.fileprovider",
+ temp
+ )
+ val intent = android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
+ setDataAndType(uri, attachment.mimeType)
+ addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
+ }
+ context.startActivity(intent)
+ } catch (e: Exception) {
+ withContext(Dispatchers.Main) {
+ android.widget.Toast.makeText(context, "Cannot open document: ${e.message}", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ }
+ }
+ }
+ ) {
+ Row(
+ modifier = Modifier.padding(12.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_translate),
+ contentDescription = "Open Document",
+ tint = MaterialTheme.colorScheme.primary,
+ modifier = Modifier.size(32.dp)
+ )
+ Spacer(modifier = Modifier.width(12.dp))
+ Column {
+ Text(
+ text = attachment.fileName,
+ style = MaterialTheme.typography.bodyMedium,
+ fontWeight = FontWeight.Bold,
+ maxLines = 1,
+ overflow = TextOverflow.Ellipsis
+ )
+ val subText = if (attachment.mimeType.startsWith("text/") || attachment.mimeType == "application/json") {
+ "Text appended inline (Tap to open)"
+ } else {
+ "Tap to open document"
+ }
+ Text(
+ text = subText,
+ style = MaterialTheme.typography.labelSmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.7f)
+ )
+ }
+ }
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun UserConversationItemPreview() {
+ AssistantTheme {
+ Column(modifier = Modifier.padding(16.dp)) {
+ ConversationItem(
+ conversation = Conversation(
+ text = "Hello! Can you help me?",
+ isMe = true,
+ category = Category.OTHER.name
+ ),
+ index = 0,
+ isSelected = false,
+ onLongClick = {},
+ isTranslateEnabled = false
+ )
+ }
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun AiConversationItemPreview() {
+ AssistantTheme {
+ Column(modifier = Modifier.padding(16.dp)) {
+ ConversationItem(
+ conversation = Conversation(
+ text = "Sure! I can help you set an alarm or make a call.",
+ isMe = false,
+ category = Category.OTHER.name
+ ),
+ index = 1,
+ isSelected = false,
+ onLongClick = {},
+ isTranslateEnabled = true
+ )
+ }
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun LoadingConversationItemPreview() {
+ AssistantTheme {
+ Column(modifier = Modifier.padding(16.dp)) {
+ ConversationItem(
+ conversation = Conversation(
+ text = "",
+ isMe = false,
+ isLoading = true
+ ),
+ index = 2,
+ isSelected = false,
+ onLongClick = {},
+ isTranslateEnabled = false
+ )
+ }
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun WeatherConversationItemPreview() {
+ AssistantTheme {
+ Column(modifier = Modifier.padding(16.dp)) {
+ ConversationItem(
+ conversation = Conversation(
+ text = "Today's weather is sunny, 25°C.",
+ isMe = false,
+ category = Category.WEATHER.name
+ ),
+ index = 3,
+ isSelected = false,
+ onLongClick = {},
+ isTranslateEnabled = false
+ )
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/screen/HandsFreeBar.kt b/app/src/main/java/com/app/assistant/ui/screen/HandsFreeBar.kt
new file mode 100644
index 0000000..a1ed76f
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/HandsFreeBar.kt
@@ -0,0 +1,563 @@
+package com.app.assistant.ui.screen
+
+import androidx.compose.animation.animateColorAsState
+import androidx.compose.animation.core.animateDpAsState
+import androidx.compose.animation.core.animateFloat
+import androidx.compose.animation.core.animateFloatAsState
+import androidx.compose.animation.core.infiniteRepeatable
+import androidx.compose.animation.core.rememberInfiniteTransition
+import androidx.compose.animation.core.FastOutSlowInEasing
+import androidx.compose.animation.core.LinearEasing
+import androidx.compose.animation.core.RepeatMode
+import androidx.compose.animation.core.tween
+import androidx.compose.animation.core.spring
+import androidx.compose.animation.core.Spring
+import androidx.compose.foundation.Canvas
+import androidx.compose.foundation.background
+import androidx.compose.foundation.layout.Arrangement
+import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.Row
+import androidx.compose.foundation.layout.fillMaxSize
+import androidx.compose.foundation.layout.fillMaxWidth
+import androidx.compose.foundation.layout.height
+import androidx.compose.foundation.layout.padding
+import androidx.compose.foundation.layout.size
+import androidx.compose.foundation.layout.width
+import androidx.compose.foundation.shape.CircleShape
+import androidx.compose.foundation.shape.RoundedCornerShape
+import androidx.compose.material.icons.Icons
+import androidx.compose.material.icons.filled.Close
+import androidx.compose.material3.CircularProgressIndicator
+import androidx.compose.material3.Icon
+import androidx.compose.material3.IconButton
+import androidx.compose.material3.MaterialTheme
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.remember
+import androidx.compose.ui.Alignment
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.graphics.Brush
+import androidx.compose.ui.graphics.Path
+import androidx.compose.ui.graphics.StrokeCap
+import androidx.compose.ui.graphics.drawscope.Stroke
+import androidx.compose.ui.graphics.drawscope.rotate
+import androidx.compose.ui.geometry.Offset
+import androidx.compose.ui.geometry.CornerRadius
+import androidx.compose.ui.geometry.Size
+import androidx.compose.ui.graphics.graphicsLayer
+import androidx.compose.ui.res.painterResource
+import androidx.compose.ui.unit.dp
+import com.app.assistant.R
+import androidx.compose.ui.tooling.preview.Preview
+import com.app.assistant.ui.theme.AssistantTheme
+
+enum class HandsFreeState {
+ STARTING,
+ LISTENING,
+ THINKING,
+ SPEAKING
+}
+
+@Composable
+fun HandsFreeBar(
+ isSpeaking: Boolean,
+ isListening: Boolean,
+ isThinking: Boolean,
+ isMicReady: Boolean,
+ onExitHandsFree: () -> Unit,
+ isMuted: Boolean = false,
+ onToggleMute: () -> Unit = {},
+ isVisionModeActive: Boolean = false,
+ onToggleVisionMode: () -> Unit = {},
+ isScreenModeActive: Boolean = false,
+ onToggleScreenMode: () -> Unit = {},
+ audioAmplitude: Float = 0f,
+ modifier: Modifier = Modifier
+) {
+ val currentState = remember(isSpeaking, isListening, isThinking, isMicReady) {
+ when {
+ isThinking -> HandsFreeState.THINKING
+ isSpeaking -> HandsFreeState.SPEAKING
+ isListening -> HandsFreeState.LISTENING
+ isMicReady -> HandsFreeState.LISTENING
+ else -> HandsFreeState.STARTING
+ }
+ }
+
+ // Dynamic state transition parameters
+ val startingAlpha by animateFloatAsState(
+ targetValue = if (currentState == HandsFreeState.STARTING) 1f else 0f,
+ animationSpec = tween(durationMillis = 350),
+ label = "startingAlpha"
+ )
+
+ val listeningProgress by animateFloatAsState(
+ targetValue = if (currentState == HandsFreeState.LISTENING) 1f else 0f,
+ animationSpec = tween(durationMillis = 350),
+ label = "listeningProgress"
+ )
+
+ val animatedAmplitude by animateFloatAsState(
+ targetValue = if (currentState == HandsFreeState.LISTENING) audioAmplitude else 0f,
+ animationSpec = spring(
+ dampingRatio = Spring.DampingRatioLowBouncy,
+ stiffness = Spring.StiffnessMedium
+ ),
+ label = "animatedAmplitude"
+ )
+
+ val thinkingProgress by animateFloatAsState(
+ targetValue = if (currentState == HandsFreeState.THINKING) 1f else 0f,
+ animationSpec = tween(durationMillis = 350),
+ label = "thinkingProgress"
+ )
+
+ val speakingProgress by animateFloatAsState(
+ targetValue = if (currentState == HandsFreeState.SPEAKING) 1f else 0f,
+ animationSpec = tween(durationMillis = 350),
+ label = "speakingProgress"
+ )
+
+ val dotsSpreadProgress by animateFloatAsState(
+ targetValue = if (currentState == HandsFreeState.THINKING || currentState == HandsFreeState.SPEAKING) 1f else 0f,
+ animationSpec = tween(durationMillis = 350),
+ label = "dotsSpreadProgress"
+ )
+
+ // Keep the orb background color and content color consistent across all states
+ val orbBackgroundColor = MaterialTheme.colorScheme.primaryContainer.copy(alpha = 0.5f)
+ val orbContentColor = MaterialTheme.colorScheme.primary
+
+ val orbWidth by animateDpAsState(
+ targetValue = when (currentState) {
+ HandsFreeState.STARTING -> 48.dp
+ HandsFreeState.LISTENING -> 72.dp
+ HandsFreeState.THINKING -> 72.dp
+ HandsFreeState.SPEAKING -> 88.dp
+ },
+ animationSpec = tween(durationMillis = 350),
+ label = "orbWidth"
+ )
+
+ val orbHeight = 48.dp
+
+ val infiniteTransition = rememberInfiniteTransition(label = "pulseTransition")
+
+ // Pulse animation for thinking outer glow ring
+ val thinkingPulseScale by infiniteTransition.animateFloat(
+ initialValue = 1.0f,
+ targetValue = 1.5f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(1200, easing = FastOutSlowInEasing),
+ repeatMode = RepeatMode.Restart
+ ),
+ label = "thinkingPulseScale"
+ )
+ val thinkingPulseAlpha by infiniteTransition.animateFloat(
+ initialValue = 0.5f,
+ targetValue = 0.0f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(1200, easing = FastOutSlowInEasing),
+ repeatMode = RepeatMode.Restart
+ ),
+ label = "thinkingPulseAlpha"
+ )
+
+ // Wave phase animation for Listening state sine wave
+ val wavePhase by infiniteTransition.animateFloat(
+ initialValue = 0f,
+ targetValue = (2 * Math.PI).toFloat(),
+ animationSpec = infiniteRepeatable(
+ animation = tween(2200, easing = LinearEasing),
+ repeatMode = RepeatMode.Restart
+ ),
+ label = "wavePhase"
+ )
+
+ // Breathing phase animation for organic pulsing
+ val breathingPhase by infiniteTransition.animateFloat(
+ initialValue = 0f,
+ targetValue = (2 * Math.PI).toFloat(),
+ animationSpec = infiniteRepeatable(
+ animation = tween(4000, easing = LinearEasing),
+ repeatMode = RepeatMode.Restart
+ ),
+ label = "breathingPhase"
+ )
+
+ // Starting state animations
+ val startingRotation by infiniteTransition.animateFloat(
+ initialValue = 0f,
+ targetValue = 360f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(1500, easing = LinearEasing),
+ repeatMode = RepeatMode.Restart
+ ),
+ label = "startingRotation"
+ )
+
+ val coreScale by infiniteTransition.animateFloat(
+ initialValue = 0.7f,
+ targetValue = 1.0f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(800, easing = FastOutSlowInEasing),
+ repeatMode = RepeatMode.Reverse
+ ),
+ label = "startingCore"
+ )
+
+ // Speaking state voice equalizer animations
+ val speakTransition = rememberInfiniteTransition(label = "speakingEqualizer")
+ val s1 by speakTransition.animateFloat(
+ initialValue = 0.15f,
+ targetValue = 0.9f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(350, easing = FastOutSlowInEasing),
+ repeatMode = RepeatMode.Reverse
+ ),
+ label = "speakBar1"
+ )
+ val s2 by speakTransition.animateFloat(
+ initialValue = 0.25f,
+ targetValue = 0.75f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(280, easing = FastOutSlowInEasing),
+ repeatMode = RepeatMode.Reverse
+ ),
+ label = "speakBar2"
+ )
+ val s3 by speakTransition.animateFloat(
+ initialValue = 0.1f,
+ targetValue = 1.0f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(420, easing = FastOutSlowInEasing),
+ repeatMode = RepeatMode.Reverse
+ ),
+ label = "speakBar3"
+ )
+ val s4 by speakTransition.animateFloat(
+ initialValue = 0.2f,
+ targetValue = 0.8f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(310, easing = FastOutSlowInEasing),
+ repeatMode = RepeatMode.Reverse
+ ),
+ label = "speakBar4"
+ )
+ val s5 by speakTransition.animateFloat(
+ initialValue = 0.15f,
+ targetValue = 0.6f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(460, easing = FastOutSlowInEasing),
+ repeatMode = RepeatMode.Reverse
+ ),
+ label = "speakBar5"
+ )
+
+ Row(
+ modifier = modifier
+ .fillMaxWidth()
+ .background(
+ color = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.85f),
+ shape = RoundedCornerShape(32.dp)
+ )
+ .padding(horizontal = 16.dp, vertical = 8.dp),
+ verticalAlignment = Alignment.CenterVertically,
+ horizontalArrangement = Arrangement.SpaceBetween
+ ) {
+ // Left actions: Camera and Share Screen
+ Row(
+ verticalAlignment = Alignment.CenterVertically,
+ horizontalArrangement = Arrangement.spacedBy(16.dp)
+ ) {
+ IconButton(
+ onClick = onToggleVisionMode
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_camera),
+ contentDescription = "Camera (Hands-free)",
+ tint = if (isVisionModeActive) MaterialTheme.colorScheme.primary else MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ }
+
+ IconButton(
+ onClick = onToggleScreenMode
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_screen_share),
+ contentDescription = "Share Screen (Hands-free)",
+ tint = if (isScreenModeActive) MaterialTheme.colorScheme.primary else MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ }
+ }
+
+ // Center State Orb
+ Box(
+ modifier = Modifier.weight(1f),
+ contentAlignment = Alignment.Center
+ ) {
+ // Outer pulsing ring (drawn behind using graphicsLayer scale so it does not affect parent height)
+ if (thinkingProgress > 0f) {
+ Box(
+ modifier = Modifier
+ .size(orbWidth, orbHeight)
+ .graphicsLayer {
+ scaleX = thinkingPulseScale
+ scaleY = thinkingPulseScale
+ alpha = thinkingPulseAlpha * thinkingProgress
+ }
+ .background(
+ color = orbBackgroundColor,
+ shape = RoundedCornerShape(24.dp)
+ )
+ )
+ }
+
+ Box(
+ modifier = Modifier
+ .size(orbWidth, orbHeight)
+ .background(color = orbBackgroundColor, shape = RoundedCornerShape(24.dp)),
+ contentAlignment = Alignment.Center
+ ) {
+ val colorPrimary = MaterialTheme.colorScheme.primary
+ val colorSecondary = MaterialTheme.colorScheme.secondary
+ val colorTertiary = MaterialTheme.colorScheme.tertiary
+
+ Canvas(modifier = Modifier.fillMaxSize()) {
+ val width = size.width
+ val height = size.height
+ val centerX = width / 2
+ val centerY = height / 2
+
+ // Helper to lerp floats
+ fun lerp(start: Float, stop: Float, fraction: Float): Float {
+ return start + fraction * (stop - start)
+ }
+
+ // A. STARTING outer ring rotating arc segment
+ if (startingAlpha > 0f) {
+ val arcSize = 32.dp.toPx()
+ rotate(degrees = startingRotation, pivot = Offset(centerX, centerY)) {
+ drawArc(
+ color = orbContentColor,
+ startAngle = 0f,
+ sweepAngle = 270f,
+ useCenter = false,
+ topLeft = Offset(centerX - arcSize / 2, centerY - arcSize / 2),
+ size = Size(arcSize, arcSize),
+ style = Stroke(width = 3.dp.toPx(), cap = StrokeCap.Round),
+ alpha = startingAlpha
+ )
+ }
+ }
+
+ // B. STARTING core dot (solid color)
+ if (startingAlpha > 0f) {
+ val startDotRadius = 6.dp.toPx() * coreScale
+ drawCircle(
+ color = orbContentColor,
+ radius = startDotRadius,
+ center = Offset(centerX, centerY),
+ alpha = startingAlpha
+ )
+ }
+
+ // C. LISTENING state morphing blob (gradient with white outline)
+ if (listeningProgress > 0f) {
+ // 1. Soft glowing background aura inside the capsule
+ val auraRadius = (height / 2) * (0.85f + 0.4f * animatedAmplitude)
+ val auraAlpha = (0.25f + 0.25f * animatedAmplitude).coerceIn(0f, 1f)
+ drawCircle(
+ brush = Brush.radialGradient(
+ colors = listOf(
+ colorTertiary.copy(alpha = auraAlpha * listeningProgress),
+ colorPrimary.copy(alpha = (auraAlpha * 0.3f) * listeningProgress),
+ Color.Transparent
+ ),
+ center = Offset(centerX, centerY),
+ radius = auraRadius
+ ),
+ radius = auraRadius,
+ center = Offset(centerX, centerY)
+ )
+
+ // 2. Slow breathing animation
+ val breathe = kotlin.math.sin(breathingPhase.toDouble()).toFloat() * 1.5f.dp.toPx() * (1f - animatedAmplitude)
+
+ // 3. Dynamic morphing blob path calculation (Single organic blob)
+ val baseRadius = lerp(6.dp.toPx(), height * 0.32f, listeningProgress) + (height * 0.08f * animatedAmplitude)
+
+ // Modest ripple amplitudes to keep the shape smooth
+ val amp1 = lerp(0.8f.dp.toPx(), 2.2f.dp.toPx(), listeningProgress) + (5.dp.toPx() * animatedAmplitude)
+ val amp2 = lerp(0.6f.dp.toPx(), 1.6f.dp.toPx(), listeningProgress) + (4.dp.toPx() * animatedAmplitude)
+ val amp3 = 3.dp.toPx() * animatedAmplitude // only when speaking to represent absorbing voice
+
+ val blobPath = Path()
+ val steps = 72
+ for (i in 0..steps) {
+ val angle = (i * 2 * Math.PI / steps).toFloat()
+ val r = baseRadius + breathe +
+ amp1 * kotlin.math.sin(2f * angle + wavePhase) +
+ amp2 * kotlin.math.cos(3f * angle - wavePhase) +
+ amp3 * kotlin.math.sin(4f * angle + wavePhase * 1.3f)
+ val x = centerX + r * kotlin.math.cos(angle)
+ val y = centerY + r * kotlin.math.sin(angle)
+ if (i == 0) {
+ blobPath.moveTo(x, y)
+ } else {
+ blobPath.lineTo(x, y)
+ }
+ }
+ blobPath.close()
+
+ val blobGradient = Brush.linearGradient(
+ colors = listOf(colorPrimary, colorTertiary, colorSecondary),
+ start = Offset(centerX - baseRadius, centerY - baseRadius),
+ end = Offset(centerX + baseRadius, centerY + baseRadius)
+ )
+
+ // Fill the blob
+ drawPath(path = blobPath, brush = blobGradient, alpha = listeningProgress)
+
+ // Thin white outline
+ drawPath(
+ path = blobPath,
+ color = Color.White.copy(alpha = (0.35f + 0.15f * animatedAmplitude) * listeningProgress),
+ style = Stroke(width = (1f + 0.5f * animatedAmplitude).dp.toPx())
+ )
+ }
+
+ // D. THINKING and SPEAKING states: 5 orbiting dots / equalizer bars
+ if (dotsSpreadProgress > 0f) {
+ val spacing = 8.dp.toPx()
+ val dotWidth = 5.dp.toPx()
+ val dotHeight = 5.dp.toPx()
+
+ val horizontalAmplitude = 5.dp.toPx()
+ val verticalAmplitude = 1.5.dp.toPx()
+
+ for (i in 0 until 5) {
+ // Equalizer height for this bar
+ val eqHeight = when (i) {
+ 0 -> 8.dp.toPx() + 16.dp.toPx() * s1
+ 1 -> 12.dp.toPx() + 14.dp.toPx() * s2
+ 2 -> 6.dp.toPx() + 22.dp.toPx() * s3
+ 3 -> 10.dp.toPx() + 18.dp.toPx() * s4
+ else -> 8.dp.toPx() + 12.dp.toPx() * s5
+ }
+
+ // Interpolate height between circle (dotHeight) and equalizer bar height
+ val currentHeight = lerp(dotHeight, eqHeight, speakingProgress)
+
+ // 3D-like orbital offsets (only active in THINKING)
+ val dotPhase = wavePhase - i * 0.6f
+ val dx = horizontalAmplitude * kotlin.math.cos(dotPhase.toDouble()).toFloat() * thinkingProgress
+ val dy = verticalAmplitude * kotlin.math.sin(dotPhase.toDouble()).toFloat() * thinkingProgress
+
+ // Interpolate home horizontal position based on spread progress
+ val homeX = centerX + (i - 2) * spacing * dotsSpreadProgress
+ val x = homeX + dx
+ val y = centerY + dy
+
+ val topLeftX = x - dotWidth / 2
+ val topLeftY = y - currentHeight / 2
+
+ drawRoundRect(
+ color = orbContentColor,
+ topLeft = Offset(topLeftX, topLeftY),
+ size = Size(dotWidth, currentHeight),
+ cornerRadius = CornerRadius(dotWidth / 2, dotWidth / 2),
+ alpha = dotsSpreadProgress
+ )
+ }
+ }
+ }
+ }
+ }
+
+ // Right actions: Mic and Close
+ Row(
+ verticalAlignment = Alignment.CenterVertically,
+ horizontalArrangement = Arrangement.spacedBy(16.dp)
+ ) {
+ IconButton(
+ onClick = onToggleMute
+ ) {
+ Icon(
+ painter = painterResource(
+ id = if (isMuted) R.drawable.ic_mic_off else R.drawable.ic_mic
+ ),
+ contentDescription = if (isMuted) "Unmute Mic" else "Mute Mic",
+ tint = MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ }
+
+ IconButton(
+ onClick = onExitHandsFree
+ ) {
+ Icon(
+ imageVector = Icons.Default.Close,
+ contentDescription = "Exit Hands-free",
+ tint = MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ }
+ }
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun HandsFreeBarStartingPreview() {
+ AssistantTheme {
+ HandsFreeBar(
+ isSpeaking = false,
+ isListening = false,
+ isThinking = false,
+ isMicReady = false,
+ onExitHandsFree = {}
+ )
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun HandsFreeBarListeningPreview() {
+ AssistantTheme {
+ HandsFreeBar(
+ isSpeaking = false,
+ isListening = true,
+ isMicReady = true,
+ isThinking = false,
+ onExitHandsFree = {}
+ )
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun HandsFreeBarThinkingPreview() {
+ AssistantTheme {
+ HandsFreeBar(
+ isSpeaking = false,
+ isListening = false,
+ isMicReady = true,
+ isThinking = true,
+ onExitHandsFree = {}
+ )
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun HandsFreeBarSpeakingPreview() {
+ AssistantTheme {
+ HandsFreeBar(
+ isSpeaking = true,
+ isListening = false,
+ isMicReady = true,
+ isThinking = false,
+ onExitHandsFree = {}
+ )
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/screen/LoadingDots.kt b/app/src/main/java/com/app/assistant/ui/screen/LoadingDots.kt
new file mode 100644
index 0000000..8cfe387
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/LoadingDots.kt
@@ -0,0 +1,101 @@
+package com.app.assistant.ui.screen
+
+import androidx.compose.animation.core.LinearEasing
+import androidx.compose.animation.core.RepeatMode
+import androidx.compose.animation.core.StartOffset
+import androidx.compose.animation.core.animateFloat
+import androidx.compose.animation.core.infiniteRepeatable
+import androidx.compose.animation.core.rememberInfiniteTransition
+import androidx.compose.animation.core.tween
+import androidx.compose.foundation.layout.Row
+import androidx.compose.foundation.layout.Spacer
+import androidx.compose.foundation.layout.offset
+import androidx.compose.foundation.layout.padding
+import androidx.compose.foundation.layout.width
+import androidx.compose.material3.MaterialTheme
+import androidx.compose.material3.Text
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.getValue
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.unit.dp
+
+import androidx.compose.ui.tooling.preview.Preview
+import com.app.assistant.ui.theme.AssistantTheme
+
+@Composable
+fun LoadingDots() {
+ val infiniteTransition = rememberInfiniteTransition()
+
+ // Animating each dot separately
+ val dot1Offset by infiniteTransition.animateFloat(
+ initialValue = 0f,
+ targetValue = -10f,
+ animationSpec =
+ infiniteRepeatable(
+ animation = tween(durationMillis = 400, easing = LinearEasing),
+ repeatMode = RepeatMode.Reverse,
+ initialStartOffset = StartOffset(0), // First dot starts immediately
+ ),
+ label = "",
+ )
+
+ val dot2Offset by infiniteTransition.animateFloat(
+ initialValue = 0f,
+ targetValue = -10f,
+ animationSpec =
+ infiniteRepeatable(
+ animation = tween(durationMillis = 400, easing = LinearEasing),
+ repeatMode = RepeatMode.Reverse,
+ initialStartOffset = StartOffset(200), // Second dot starts with a delay
+ ),
+ label = "",
+ )
+
+ val dot3Offset by infiniteTransition.animateFloat(
+ initialValue = 0f,
+ targetValue = -10f,
+ animationSpec =
+ infiniteRepeatable(
+ animation = tween(durationMillis = 400, easing = LinearEasing),
+ repeatMode = RepeatMode.Reverse,
+ initialStartOffset = StartOffset(400), // Third dot starts with a larger delay
+ ),
+ label = "",
+ )
+
+ // Unicode character for big dot (⏺ or ● for larger circles)
+ val dotChar = "\u25CF"
+
+ Row(modifier = Modifier.padding(16.dp)) {
+ // First dot
+ Text(
+ text = dotChar,
+ modifier = Modifier.offset(y = dot1Offset.dp),
+ style = MaterialTheme.typography.bodyLarge,
+ )
+ Spacer(modifier = Modifier.width(4.dp))
+
+ // Second dot
+ Text(
+ text = dotChar,
+ modifier = Modifier.offset(y = dot2Offset.dp),
+ style = MaterialTheme.typography.bodyLarge,
+ )
+ Spacer(modifier = Modifier.width(4.dp))
+
+ // Third dot
+ Text(
+ text = dotChar,
+ modifier = Modifier.offset(y = dot3Offset.dp),
+ style = MaterialTheme.typography.bodyLarge,
+ )
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun LoadingDotsPreview() {
+ AssistantTheme {
+ LoadingDots()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/screen/MarkdownText.kt b/app/src/main/java/com/app/assistant/ui/screen/MarkdownText.kt
new file mode 100644
index 0000000..8c9088d
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/MarkdownText.kt
@@ -0,0 +1,72 @@
+package com.app.assistant.ui.screen
+
+import androidx.compose.material3.Text
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.remember
+import androidx.compose.ui.Modifier
+import androidx.compose.foundation.layout.padding
+import androidx.compose.ui.unit.dp
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.text.AnnotatedString
+import androidx.compose.ui.text.SpanStyle
+import androidx.compose.ui.text.font.FontFamily
+import androidx.compose.ui.text.font.FontStyle
+import androidx.compose.ui.text.font.FontWeight
+import androidx.compose.ui.text.style.TextDecoration
+import androidx.compose.ui.unit.sp
+import org.commonmark.node.BlockQuote
+import org.commonmark.node.BulletList
+import org.commonmark.node.Code
+import org.commonmark.node.Emphasis
+import org.commonmark.node.FencedCodeBlock
+import org.commonmark.node.HardLineBreak
+import org.commonmark.node.Heading
+import org.commonmark.node.IndentedCodeBlock
+import org.commonmark.node.Link
+import org.commonmark.node.ListItem
+import org.commonmark.node.Node
+import org.commonmark.node.OrderedList
+import org.commonmark.node.Paragraph
+import org.commonmark.node.SoftLineBreak
+import org.commonmark.node.StrongEmphasis
+import org.commonmark.node.ThematicBreak
+import org.commonmark.parser.Parser
+import org.commonmark.node.Text as CText
+
+import androidx.compose.ui.tooling.preview.Preview
+import com.app.assistant.ui.theme.AssistantTheme
+
+@Composable
+fun MarkdownText(
+ modifier: Modifier = Modifier,
+ markdown: String,
+ linkColor: Color = androidx.compose.material3.MaterialTheme.colorScheme.primary,
+) {
+ val annotatedText =
+ remember(markdown, linkColor) {
+ com.app.assistant.util.MarkdownUtils.parseMarkdownToAnnotatedString(markdown, linkColor)
+ }
+ Text(
+ text = annotatedText,
+ modifier = modifier,
+ )
+}
+
+@Preview(showBackground = true)
+@Composable
+fun MarkdownTextPreview() {
+ AssistantTheme {
+ MarkdownText(
+ modifier = Modifier.padding(16.dp),
+ markdown = """
+ # Markdown Preview
+ This is a **bold** text and *italic* text.
+ - List item 1
+ - List item 2
+
+ [Link to Google](https://google.com)
+ """.trimIndent()
+ )
+ }
+}
+
diff --git a/app/src/main/java/com/app/assistant/ui/screen/QrScannerDialog.kt b/app/src/main/java/com/app/assistant/ui/screen/QrScannerDialog.kt
new file mode 100644
index 0000000..3377a77
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/QrScannerDialog.kt
@@ -0,0 +1,418 @@
+package com.app.assistant.ui.screen
+
+import android.Manifest
+import android.content.pm.PackageManager
+import android.util.Log
+import android.util.Size
+import androidx.activity.compose.rememberLauncherForActivityResult
+import androidx.activity.result.contract.ActivityResultContracts
+import androidx.camera.core.CameraSelector
+import androidx.camera.core.ExperimentalGetImage
+import androidx.camera.core.ImageAnalysis
+import androidx.camera.core.ImageProxy
+import androidx.camera.core.Preview
+import androidx.camera.lifecycle.ProcessCameraProvider
+import androidx.camera.view.PreviewView
+import androidx.compose.animation.core.*
+import androidx.compose.foundation.Canvas
+import androidx.compose.foundation.background
+import androidx.compose.foundation.clickable
+import androidx.compose.foundation.layout.*
+import androidx.compose.foundation.shape.RoundedCornerShape
+import androidx.compose.material.icons.Icons
+import androidx.compose.material.icons.filled.Close
+import androidx.compose.material.icons.filled.Warning
+import androidx.compose.material3.*
+import androidx.compose.runtime.*
+import androidx.compose.ui.Alignment
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.geometry.CornerRadius
+import androidx.compose.ui.geometry.Offset
+import androidx.compose.ui.geometry.Rect
+import androidx.compose.ui.geometry.RoundRect
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.graphics.Path
+import androidx.compose.ui.graphics.PathOperation
+import androidx.compose.ui.graphics.drawscope.Stroke
+import androidx.compose.ui.platform.LocalContext
+import androidx.compose.ui.platform.LocalLifecycleOwner
+import androidx.compose.ui.platform.LocalUriHandler
+import androidx.compose.ui.text.font.FontWeight
+import androidx.compose.ui.text.style.TextAlign
+import androidx.compose.ui.text.style.TextDecoration
+import androidx.compose.ui.unit.dp
+import androidx.compose.ui.unit.sp
+import androidx.compose.ui.viewinterop.AndroidView
+import androidx.compose.ui.window.Dialog
+import androidx.compose.ui.window.DialogProperties
+import androidx.core.content.ContextCompat
+import com.google.mlkit.vision.barcode.BarcodeScannerOptions
+import com.google.mlkit.vision.barcode.BarcodeScanning
+import com.google.mlkit.vision.barcode.common.Barcode
+import com.google.mlkit.vision.common.InputImage
+import java.util.concurrent.Executors
+
+// The URL where users can securely import and export their keys as a QR code
+const val EXPORT_KEYS_WEBSITE_URL = "https://souravanand001.github.io/ai-assistant-android/"
+
+@Composable
+fun QrScannerDialog(
+ onDismissRequest: () -> Unit,
+ onCodeScanned: (String) -> Unit
+) {
+ val context = LocalContext.current
+ var hasCameraPermission by remember {
+ mutableStateOf(
+ ContextCompat.checkSelfPermission(context, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED
+ )
+ }
+
+ val permissionLauncher = rememberLauncherForActivityResult(
+ contract = ActivityResultContracts.RequestPermission(),
+ onResult = { granted ->
+ hasCameraPermission = granted
+ }
+ )
+
+ LaunchedEffect(Unit) {
+ if (!hasCameraPermission) {
+ permissionLauncher.launch(Manifest.permission.CAMERA)
+ }
+ }
+
+ Dialog(
+ onDismissRequest = onDismissRequest,
+ properties = DialogProperties(
+ usePlatformDefaultWidth = false,
+ dismissOnBackPress = true,
+ dismissOnClickOutside = false
+ )
+ ) {
+ Box(
+ modifier = Modifier
+ .fillMaxSize()
+ .background(Color.Black)
+ ) {
+ if (hasCameraPermission) {
+ ScannerCameraPreview(
+ onCodeScanned = onCodeScanned,
+ onDismissRequest = onDismissRequest
+ )
+ } else {
+ PermissionDeniedContent(
+ onRequestPermission = { permissionLauncher.launch(Manifest.permission.CAMERA) },
+ onClose = onDismissRequest
+ )
+ }
+ }
+ }
+}
+
+@Composable
+fun ScannerCameraPreview(
+ onCodeScanned: (String) -> Unit,
+ onDismissRequest: () -> Unit
+) {
+ val context = LocalContext.current
+ val lifecycleOwner = LocalLifecycleOwner.current
+ val uriHandler = LocalUriHandler.current
+ var isScanned by remember { mutableStateOf(false) }
+
+ val previewView = remember {
+ PreviewView(context).apply {
+ scaleType = PreviewView.ScaleType.FILL_CENTER
+ }
+ }
+
+ // Camera Lifecycle & Setup
+ DisposableEffect(lifecycleOwner) {
+ val cameraProviderFuture = ProcessCameraProvider.getInstance(context)
+ val executor = Executors.newSingleThreadExecutor()
+
+ cameraProviderFuture.addListener({
+ try {
+ val cameraProvider = cameraProviderFuture.get()
+ val preview = Preview.Builder().build().apply {
+ setSurfaceProvider(previewView.surfaceProvider)
+ }
+
+ // Analyze frames with ML Kit Barcode Scanner
+ val imageAnalysis = ImageAnalysis.Builder()
+ .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
+ .build()
+
+ imageAnalysis.setAnalyzer(
+ executor,
+ QrCodeAnalyzer { result ->
+ if (!isScanned) {
+ isScanned = true
+ // Trigger callback on main thread
+ ContextCompat.getMainExecutor(context).execute {
+ onCodeScanned(result)
+ }
+ }
+ }
+ )
+
+ val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA
+ cameraProvider.unbindAll()
+ cameraProvider.bindToLifecycle(
+ lifecycleOwner,
+ cameraSelector,
+ preview,
+ imageAnalysis
+ )
+
+ } catch (e: Exception) {
+ Log.e("QrScannerDialog", "CameraX UseCase binding failed", e)
+ }
+ }, ContextCompat.getMainExecutor(context))
+
+ onDispose {
+ try {
+ if (cameraProviderFuture.isDone) {
+ val cameraProvider = cameraProviderFuture.get()
+ cameraProvider.unbindAll()
+ }
+ } catch (e: Exception) {
+ Log.e("QrScannerDialog", "Error unbinding CameraX", e)
+ }
+ executor.shutdown()
+ }
+ }
+
+ // Camera Feed Underlay
+ AndroidView(
+ factory = { previewView },
+ modifier = Modifier.fillMaxSize()
+ )
+
+ // Scanner overlay (Canvas with square cutout + neon green frame + scanning line)
+ val infiniteTransition = rememberInfiniteTransition(label = "laser")
+ val laserPosition by infiniteTransition.animateFloat(
+ initialValue = 0f,
+ targetValue = 1f,
+ animationSpec = infiniteRepeatable(
+ animation = tween(durationMillis = 2500, easing = LinearEasing),
+ repeatMode = RepeatMode.Reverse
+ ),
+ label = "laserPosition"
+ )
+
+ Canvas(modifier = Modifier.fillMaxSize()) {
+ val canvasWidth = size.width
+ val canvasHeight = size.height
+ val boxWidth = canvasWidth * 0.7f
+ val boxHeight = boxWidth // Square viewfinder
+
+ val left = (canvasWidth - boxWidth) / 2
+ val top = (canvasHeight - boxHeight) / 2
+
+ // Draw transparent black mask with cutout
+ val path = Path().apply {
+ addRect(Rect(0f, 0f, canvasWidth, canvasHeight))
+ }
+ val cutoutPath = Path().apply {
+ addRoundRect(
+ RoundRect(
+ left = left,
+ top = top,
+ right = left + boxWidth,
+ bottom = top + boxHeight,
+ cornerRadius = CornerRadius(24.dp.toPx(), 24.dp.toPx())
+ )
+ )
+ }
+
+ val finalPath = Path.combine(PathOperation.Difference, path, cutoutPath)
+ drawPath(finalPath, color = Color.Black.copy(alpha = 0.65f))
+
+ // Viewfinder Frame Outline
+ drawRoundRect(
+ color = Color(0xFF00E676), // Neon Green
+ topLeft = Offset(left, top),
+ size = androidx.compose.ui.geometry.Size(boxWidth, boxHeight),
+ cornerRadius = CornerRadius(24.dp.toPx(), 24.dp.toPx()),
+ style = Stroke(width = 3.dp.toPx())
+ )
+
+ // Viewfinder Target Breathing Line (Red Laser)
+ val laserY = top + (boxHeight * laserPosition)
+ drawLine(
+ color = Color(0xFFFF1744), // Bright Red
+ start = Offset(left + 15.dp.toPx(), laserY),
+ end = Offset(left + boxWidth - 15.dp.toPx(), laserY),
+ strokeWidth = 2.dp.toPx()
+ )
+ }
+
+ // UI Controls Overlay
+ Box(
+ modifier = Modifier
+ .fillMaxSize()
+ .padding(16.dp)
+ ) {
+ // Close Button
+ IconButton(
+ onClick = onDismissRequest,
+ modifier = Modifier
+ .align(Alignment.TopStart)
+ .statusBarsPadding()
+ .background(Color.Black.copy(alpha = 0.5f), RoundedCornerShape(50))
+ ) {
+ Icon(
+ imageVector = Icons.Default.Close,
+ contentDescription = "Close Scanner",
+ tint = Color.White
+ )
+ }
+
+ // Instructions Card
+ Card(
+ modifier = Modifier
+ .align(Alignment.TopCenter)
+ .padding(top = 70.dp)
+ .statusBarsPadding()
+ .fillMaxWidth(0.9f),
+ colors = CardDefaults.cardColors(
+ containerColor = Color(0xFF1E1E2C).copy(alpha = 0.9f)
+ ),
+ shape = RoundedCornerShape(16.dp),
+ elevation = CardDefaults.cardElevation(defaultElevation = 8.dp)
+ ) {
+ Column(
+ modifier = Modifier.padding(16.dp),
+ horizontalAlignment = Alignment.CenterHorizontally
+ ) {
+ Text(
+ text = "Import API Keys Safely",
+ color = Color.White,
+ fontWeight = FontWeight.Bold,
+ fontSize = 16.sp,
+ textAlign = TextAlign.Center
+ )
+ Spacer(modifier = Modifier.height(6.dp))
+ Text(
+ text = "To import your API keys securely, visit this URL on your desktop/other device:",
+ color = Color.LightGray,
+ fontSize = 12.sp,
+ textAlign = TextAlign.Center
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+ Text(
+ text = EXPORT_KEYS_WEBSITE_URL,
+ color = Color(0xFF29B6F6), // Sky Blue Link Color
+ fontWeight = FontWeight.SemiBold,
+ fontSize = 13.sp,
+ textDecoration = TextDecoration.Underline,
+ textAlign = TextAlign.Center,
+ modifier = Modifier
+ .clickable {
+ try {
+ uriHandler.openUri(EXPORT_KEYS_WEBSITE_URL)
+ } catch (e: Exception) {
+ Log.e("QrScannerDialog", "Failed to open URI", e)
+ }
+ }
+ .padding(4.dp)
+ )
+ Spacer(modifier = Modifier.height(4.dp))
+ Text(
+ text = "Align the QR code on the screen within the frame to scan.",
+ color = Color.Gray,
+ fontSize = 11.sp,
+ textAlign = TextAlign.Center
+ )
+ }
+ }
+ }
+}
+
+@Composable
+fun PermissionDeniedContent(
+ onRequestPermission: () -> Unit,
+ onClose: () -> Unit
+) {
+ Column(
+ modifier = Modifier
+ .fillMaxSize()
+ .padding(24.dp),
+ horizontalAlignment = Alignment.CenterHorizontally,
+ verticalArrangement = Arrangement.Center
+ ) {
+ Icon(
+ imageVector = Icons.Default.Warning,
+ contentDescription = "Warning",
+ tint = Color(0xFFFFB300),
+ modifier = Modifier.size(64.dp)
+ )
+ Spacer(modifier = Modifier.height(16.dp))
+ Text(
+ text = "Camera Permission Required",
+ color = Color.White,
+ fontWeight = FontWeight.Bold,
+ fontSize = 20.sp,
+ textAlign = TextAlign.Center
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+ Text(
+ text = "We need access to your camera to scan QR codes and safely import API keys.",
+ color = Color.LightGray,
+ fontSize = 14.sp,
+ textAlign = TextAlign.Center
+ )
+ Spacer(modifier = Modifier.height(24.dp))
+ Button(
+ onClick = onRequestPermission,
+ colors = ButtonDefaults.buttonColors(
+ containerColor = Color(0xFF00E676),
+ contentColor = Color.Black
+ ),
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ Text("Grant Permission", fontWeight = FontWeight.Bold)
+ }
+ Spacer(modifier = Modifier.height(12.dp))
+ TextButton(onClick = onClose) {
+ Text("Cancel", color = Color.White)
+ }
+ }
+}
+
+class QrCodeAnalyzer(
+ private val onQrCodeScanned: (String) -> Unit
+) : ImageAnalysis.Analyzer {
+
+ private val options = BarcodeScannerOptions.Builder()
+ .setBarcodeFormats(Barcode.FORMAT_QR_CODE)
+ .build()
+
+ private val scanner = BarcodeScanning.getClient(options)
+
+ @ExperimentalGetImage
+ override fun analyze(imageProxy: ImageProxy) {
+ val mediaImage = imageProxy.image
+ if (mediaImage != null) {
+ val image = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
+ scanner.process(image)
+ .addOnSuccessListener { barcodes ->
+ for (barcode in barcodes) {
+ val rawValue = barcode.rawValue
+ if (rawValue != null && rawValue.isNotBlank()) {
+ onQrCodeScanned(rawValue)
+ break
+ }
+ }
+ }
+ .addOnFailureListener { e ->
+ Log.e("QrCodeAnalyzer", "Barcode scanning failed", e)
+ }
+ .addOnCompleteListener {
+ imageProxy.close()
+ }
+ } else {
+ imageProxy.close()
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/screen/SettingsScreen.kt b/app/src/main/java/com/app/assistant/ui/screen/SettingsScreen.kt
new file mode 100644
index 0000000..66bf327
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/SettingsScreen.kt
@@ -0,0 +1,2244 @@
+package com.app.assistant.ui.screen
+
+import androidx.activity.compose.BackHandler
+import androidx.compose.foundation.clickable
+import androidx.compose.foundation.layout.Arrangement
+import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.Column
+import androidx.compose.foundation.layout.Row
+import androidx.compose.foundation.layout.Spacer
+import androidx.compose.foundation.layout.fillMaxSize
+import androidx.compose.foundation.layout.fillMaxWidth
+import androidx.compose.foundation.layout.height
+import androidx.compose.foundation.layout.padding
+import androidx.compose.foundation.layout.size
+import androidx.compose.foundation.layout.width
+import androidx.compose.foundation.rememberScrollState
+import androidx.compose.foundation.verticalScroll
+import androidx.compose.material.icons.Icons
+import androidx.compose.material.icons.filled.Check
+import androidx.compose.material.icons.filled.Close
+import androidx.compose.material.icons.filled.Delete
+import androidx.compose.material.icons.filled.Info
+import androidx.compose.foundation.BorderStroke
+import androidx.compose.foundation.shape.RoundedCornerShape
+import androidx.compose.foundation.isSystemInDarkTheme
+import androidx.compose.material3.Button
+import androidx.compose.material3.ButtonDefaults
+import androidx.compose.material3.Card
+import androidx.compose.material3.CardDefaults
+import androidx.compose.material3.CircularProgressIndicator
+import androidx.compose.material3.DropdownMenu
+import androidx.compose.material3.DropdownMenuItem
+import androidx.compose.material3.ExperimentalMaterial3Api
+import androidx.compose.material3.Icon
+import androidx.compose.material3.IconButton
+import androidx.compose.material3.MaterialTheme
+import androidx.compose.material3.OutlinedButton
+import androidx.compose.material3.OutlinedTextField
+import androidx.compose.material3.Scaffold
+import androidx.compose.material3.Surface
+import androidx.compose.material3.Text
+import androidx.compose.material3.TopAppBar
+import androidx.compose.material3.TopAppBarDefaults
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.LaunchedEffect
+import androidx.compose.runtime.collectAsState
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.mutableStateOf
+import androidx.compose.runtime.remember
+import androidx.compose.runtime.saveable.rememberSaveable
+import androidx.compose.runtime.setValue
+import androidx.compose.ui.Alignment
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.platform.LocalContext
+import androidx.compose.ui.res.painterResource
+import androidx.compose.ui.res.stringResource
+import androidx.compose.ui.text.font.FontWeight
+import androidx.compose.ui.unit.dp
+import com.app.assistant.R
+import com.app.assistant.llm.LlmProvider
+import com.app.assistant.viewmodel.SettingsViewModel
+import com.app.assistant.viewmodel.VerificationState
+
+import androidx.compose.material3.Switch
+import androidx.compose.material3.RadioButton
+import com.app.assistant.speech.SttMode
+import com.app.assistant.config.SpeechConfig
+import com.app.assistant.viewmodel.DownloadState
+import androidx.activity.compose.rememberLauncherForActivityResult
+import androidx.activity.result.contract.ActivityResultContracts
+import androidx.compose.ui.tooling.preview.Preview
+import com.app.assistant.ui.theme.AssistantTheme
+import androidx.compose.material3.SheetState
+import androidx.compose.material3.rememberModalBottomSheetState
+import androidx.compose.material3.ModalBottomSheet
+import androidx.compose.foundation.lazy.LazyColumn
+import androidx.compose.foundation.lazy.items
+import android.app.role.RoleManager
+import android.content.Intent
+import android.content.Context
+import android.provider.Settings
+import android.os.Build
+import androidx.compose.runtime.DisposableEffect
+import androidx.compose.ui.platform.LocalLifecycleOwner
+import androidx.lifecycle.Lifecycle
+import androidx.lifecycle.LifecycleEventObserver
+
+@OptIn(ExperimentalMaterial3Api::class)
+@Composable
+fun SettingsScreen(
+ settingsViewModel: SettingsViewModel,
+ onBack: () -> Unit,
+ onTranslationEnabledChange: (Boolean) -> Unit,
+ onLanguageSelected: (String) -> Unit,
+) {
+ val initialYoutubeKey = remember { settingsViewModel.loadYoutubeKey() }
+ val initialEdgeTtsKey = remember { settingsViewModel.loadEdgeTtsSubscriptionKey() }
+ val initialChatKey = remember { settingsViewModel.loadChatKey() }
+ val initialProvider = remember { settingsViewModel.loadLlmProvider() }
+ val initialModel = remember { settingsViewModel.loadLlmModel() }
+ val initialCustomUrl = remember { settingsViewModel.loadLlmCustomUrl() }
+ val initialCustomHeaders = remember { settingsViewModel.loadLlmCustomHeaders() }
+ val initialCustomResponsePath = remember { settingsViewModel.loadLlmCustomResponsePath() }
+ val initialCustomRequestTemplate = remember { settingsViewModel.loadLlmCustomRequestTemplate() }
+ val initialCustomMessageFormat = remember { settingsViewModel.loadLlmCustomMessageFormat() }
+ val initialCustomSystemRole = remember { settingsViewModel.loadLlmCustomSystemRole() }
+ val initialCustomUserRole = remember { settingsViewModel.loadLlmCustomUserRole() }
+ val initialCustomAssistantRole = remember { settingsViewModel.loadLlmCustomAssistantRole() }
+ val initialSttMode = remember { settingsViewModel.loadSttMode() }
+ val initialTtsMode = remember { settingsViewModel.loadTtsMode() }
+
+ val isTtsInstalled by settingsViewModel.isTtsModelInstalled.collectAsState()
+ val verificationState by settingsViewModel.verificationState.collectAsState()
+ val fetchedModelsMap by settingsViewModel.fetchedModels.collectAsState()
+ val isFetchingModels by settingsViewModel.isFetchingModels.collectAsState()
+ val modelFetchError by settingsViewModel.modelFetchError.collectAsState()
+ val isModelInstalled by settingsViewModel.isModelInstalled.collectAsState()
+ val modelDownloadState by settingsViewModel.modelDownloadState.collectAsState()
+ val ttsDownloadState by settingsViewModel.ttsModelDownloadState.collectAsState()
+
+ val isTranslationEnabled by settingsViewModel.isTranslationEnabled.collectAsState()
+ val activeLanguageCode by settingsViewModel.activeLanguageCode.collectAsState()
+ val downloadedTranslationLanguages by settingsViewModel.downloadedTranslationLanguages.collectAsState()
+ val downloadingTranslationLanguages by settingsViewModel.downloadingTranslationLanguages.collectAsState()
+ val translationLanguages = settingsViewModel.translationLanguages
+ val isDefaultAssistant by settingsViewModel.isDefaultAssistant.collectAsState()
+
+ SettingsScreenContent(
+ settingsViewModel = settingsViewModel,
+ initialYoutubeKey = initialYoutubeKey,
+ initialEdgeTtsKey = initialEdgeTtsKey,
+ initialChatKey = initialChatKey,
+ initialProvider = initialProvider,
+ initialModel = initialModel,
+ initialCustomUrl = initialCustomUrl,
+ initialCustomHeaders = initialCustomHeaders,
+ initialCustomResponsePath = initialCustomResponsePath,
+ initialCustomRequestTemplate = initialCustomRequestTemplate,
+ initialCustomMessageFormat = initialCustomMessageFormat,
+ initialCustomSystemRole = initialCustomSystemRole,
+ initialCustomUserRole = initialCustomUserRole,
+ initialCustomAssistantRole = initialCustomAssistantRole,
+ initialSttMode = initialSttMode,
+ initialTtsMode = initialTtsMode,
+ isTtsInstalled = isTtsInstalled,
+ verificationState = verificationState,
+ fetchedModelsMap = fetchedModelsMap,
+ isFetchingModels = isFetchingModels,
+ modelFetchError = modelFetchError,
+ isModelInstalled = isModelInstalled,
+ modelDownloadState = modelDownloadState,
+ ttsDownloadState = ttsDownloadState,
+ isTranslationEnabled = isTranslationEnabled,
+ activeLanguageCode = activeLanguageCode,
+ downloadedTranslationLanguages = downloadedTranslationLanguages,
+ downloadingTranslationLanguages = downloadingTranslationLanguages,
+ translationLanguages = translationLanguages,
+ isDefaultAssistant = isDefaultAssistant,
+ onBack = onBack,
+ onResetVerificationState = { settingsViewModel.resetVerificationState() },
+ onFetchModelsForProvider = { provider, key, url, headers ->
+ settingsViewModel.fetchModelsForProvider(provider, key, url, headers)
+ },
+ onVerifyModelAndSaveCapabilities = { provider, modelName, key, url, headers, respPath, reqTempl, msgFormat, sysRole, userRole, assRole ->
+ settingsViewModel.verifyModelAndSaveCapabilities(
+ provider, modelName, key, url, headers, respPath, reqTempl, msgFormat, sysRole, userRole, assRole
+ )
+ },
+ onStartModelDownload = { settingsViewModel.startModelDownload() },
+ onDeleteModel = { settingsViewModel.deleteModel() },
+ onCancelModelDownload = { settingsViewModel.cancelModelDownload() },
+ onStartTtsModelDownload = { settingsViewModel.startTtsModelDownload() },
+ onDeleteTtsModel = { settingsViewModel.deleteTtsModel() },
+ onCancelTtsModelDownload = { settingsViewModel.cancelTtsModelDownload() },
+ onTranslationEnabledChange = onTranslationEnabledChange,
+ onLanguageSelected = onLanguageSelected,
+ onDownloadTranslationModel = { settingsViewModel.downloadTranslationModel(it) },
+ onDeleteTranslationModel = { settingsViewModel.deleteTranslationModel(it) }
+ )
+}
+
+@OptIn(ExperimentalMaterial3Api::class)
+@Composable
+fun SettingsScreenContent(
+ settingsViewModel: SettingsViewModel,
+ initialYoutubeKey: String,
+ initialEdgeTtsKey: String,
+ initialChatKey: String,
+ initialProvider: LlmProvider,
+ initialModel: String,
+ initialCustomUrl: String,
+ initialCustomHeaders: String,
+ initialCustomResponsePath: String,
+ initialCustomRequestTemplate: String,
+ initialCustomMessageFormat: String,
+ initialCustomSystemRole: String,
+ initialCustomUserRole: String,
+ initialCustomAssistantRole: String,
+ initialSttMode: SttMode,
+ initialTtsMode: com.app.assistant.tts.TtsMode,
+ isTtsInstalled: Boolean,
+ verificationState: VerificationState,
+ fetchedModelsMap: Map>,
+ isFetchingModels: Boolean,
+ modelFetchError: String?,
+ isModelInstalled: Boolean,
+ modelDownloadState: DownloadState,
+ ttsDownloadState: DownloadState,
+ isTranslationEnabled: Boolean,
+ activeLanguageCode: String,
+ downloadedTranslationLanguages: Set,
+ downloadingTranslationLanguages: Set,
+ translationLanguages: List>,
+ isDefaultAssistant: Boolean,
+ onBack: () -> Unit,
+ onResetVerificationState: () -> Unit,
+ onFetchModelsForProvider: (LlmProvider, String, String, String) -> Unit,
+ onVerifyModelAndSaveCapabilities: (
+ LlmProvider, String, String, String, String, String, String, String, String, String, String
+ ) -> Unit,
+ onStartModelDownload: () -> Unit,
+ onDeleteModel: () -> Unit,
+ onCancelModelDownload: () -> Unit,
+ onStartTtsModelDownload: () -> Unit,
+ onDeleteTtsModel: () -> Unit,
+ onCancelTtsModelDownload: () -> Unit,
+ onTranslationEnabledChange: (Boolean) -> Unit,
+ onLanguageSelected: (String) -> Unit,
+ onDownloadTranslationModel: (String) -> Unit,
+ onDeleteTranslationModel: (String) -> Unit
+) {
+ var apiKey1 by rememberSaveable { mutableStateOf(initialYoutubeKey) }
+ var edgeTtsKey by rememberSaveable { mutableStateOf(initialEdgeTtsKey) }
+ var apiKey2 by rememberSaveable { mutableStateOf(initialChatKey) }
+ var provider by rememberSaveable { mutableStateOf(initialProvider) }
+ var model by rememberSaveable { mutableStateOf(initialModel) }
+ var sttMode by rememberSaveable { mutableStateOf(initialSttMode) }
+ var ttsMode by rememberSaveable { mutableStateOf(initialTtsMode) }
+
+ var showQrScanner by remember { mutableStateOf(false) }
+ var targetKeyField by remember { mutableStateOf(null) } // "chat", "youtube", "edge_tts"
+
+
+ var customUrl by rememberSaveable { mutableStateOf(initialCustomUrl) }
+ var customHeaders by rememberSaveable { mutableStateOf(initialCustomHeaders) }
+ var customResponsePath by rememberSaveable { mutableStateOf(initialCustomResponsePath) }
+ var customRequestTemplate by rememberSaveable { mutableStateOf(initialCustomRequestTemplate) }
+ var customMessageFormat by rememberSaveable { mutableStateOf(initialCustomMessageFormat) }
+ var customSystemRole by rememberSaveable { mutableStateOf(initialCustomSystemRole) }
+ var customUserRole by rememberSaveable { mutableStateOf(initialCustomUserRole) }
+ var customAssistantRole by rememberSaveable { mutableStateOf(initialCustomAssistantRole) }
+
+ var showTranslationSheet by remember { mutableStateOf(false) }
+
+
+
+ LaunchedEffect(isTtsInstalled) {
+ if (ttsMode == com.app.assistant.tts.TtsMode.OFFLINE) {
+ if (!isTtsInstalled) {
+ ttsMode = com.app.assistant.tts.TtsMode.NATIVE
+ settingsViewModel.updateTtsMode(com.app.assistant.tts.TtsMode.NATIVE)
+ }
+ }
+ }
+
+ val lifecycleOwner = LocalLifecycleOwner.current
+ DisposableEffect(lifecycleOwner) {
+ val observer = LifecycleEventObserver { _, event ->
+ if (event == Lifecycle.Event.ON_RESUME) {
+ settingsViewModel.checkDefaultAssistantStatus()
+ }
+ }
+ lifecycleOwner.lifecycle.addObserver(observer)
+ onDispose {
+ lifecycleOwner.lifecycle.removeObserver(observer)
+ }
+ }
+
+ var expanded by remember { mutableStateOf(false) }
+
+ var modelDropdownExpanded by remember { mutableStateOf(false) }
+
+ val context = LocalContext.current
+ val versionName = remember(context) {
+ try {
+ val packageInfo = context.packageManager.getPackageInfo(context.packageName, 0)
+ packageInfo.versionName ?: "1.0.0"
+ } catch (e: Exception) {
+ "1.0.0"
+ }
+ }
+
+ LaunchedEffect(Unit) {
+ onResetVerificationState()
+ val currentKey = initialChatKey
+ if (currentKey.isNotBlank() || initialProvider == LlmProvider.OLLAMA || initialProvider == LlmProvider.OPEN_ROUTER) {
+ onFetchModelsForProvider(
+ initialProvider,
+ currentKey,
+ initialCustomUrl,
+ initialCustomHeaders
+ )
+ }
+ }
+
+ BackHandler(enabled = true) {
+ onBack()
+ }
+
+ Scaffold(
+ topBar = {
+ TopAppBar(
+ title = {
+ Text(
+ text = stringResource(id = R.string.settings_title),
+ fontWeight = FontWeight.ExtraBold,
+ style = MaterialTheme.typography.titleLarge
+ )
+ },
+ navigationIcon = {
+ IconButton(onClick = onBack) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_back),
+ contentDescription = "Back",
+ tint = MaterialTheme.colorScheme.onBackground
+ )
+ }
+ },
+ colors = TopAppBarDefaults.topAppBarColors(
+ containerColor = MaterialTheme.colorScheme.background,
+ titleContentColor = MaterialTheme.colorScheme.onBackground
+ )
+ )
+ }
+ ) { innerPadding ->
+ Column(
+ modifier = Modifier
+ .fillMaxSize()
+ .padding(innerPadding)
+ .padding(horizontal = 16.dp)
+ .verticalScroll(rememberScrollState())
+ ) {
+ Spacer(modifier = Modifier.height(8.dp))
+
+ // CARD 1: AI MODEL SETTINGS
+ Card(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 8.dp),
+ shape = RoundedCornerShape(16.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.2f)
+ ),
+ elevation = CardDefaults.cardElevation(defaultElevation = 0.dp)
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ SettingsSectionHeader(
+ title = "AI Model Provider",
+ description = "Configure the main language model for chat",
+ icon = R.drawable.ic_settings
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+
+ Box(modifier = Modifier.fillMaxWidth()) {
+ OutlinedTextField(
+ value = provider.displayName,
+ onValueChange = {},
+ readOnly = true,
+ label = { Text(stringResource(id = R.string.llm_provider)) },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth(),
+ trailingIcon = {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_arrow_drop_down),
+ contentDescription = null
+ )
+ }
+ )
+ Box(
+ modifier = Modifier
+ .matchParentSize()
+ .clickable { expanded = true }
+ )
+ DropdownMenu(
+ expanded = expanded,
+ onDismissRequest = { expanded = false },
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ LlmProvider.entries.forEach { p ->
+ DropdownMenuItem(
+ text = { Text(p.displayName) },
+ onClick = {
+ if (provider != p) {
+ provider = p
+ settingsViewModel.updateLlmProvider(p)
+ onResetVerificationState()
+ if (p != LlmProvider.CUSTOM) {
+ model = p.defaultModel
+ settingsViewModel.updateLlmModel(p.defaultModel)
+ val currentKey = apiKey2
+ if (currentKey.isNotBlank() || p == LlmProvider.OLLAMA || p == LlmProvider.OPEN_ROUTER) {
+ onFetchModelsForProvider(p, currentKey, "", "")
+ }
+ } else {
+ model = p.defaultModel
+ settingsViewModel.updateLlmModel(p.defaultModel)
+ if (customUrl.isBlank()) {
+ customUrl = p.config.url
+ settingsViewModel.updateLlmCustomUrl(p.config.url)
+ }
+ if (customHeaders.isBlank()) {
+ customHeaders = "{\"Authorization\": \"Bearer {{API_KEY}}\", \"Content-Type\": \"application/json\"}"
+ settingsViewModel.updateLlmCustomHeaders(customHeaders)
+ }
+ if (customResponsePath.isBlank()) {
+ customResponsePath = p.config.responsePath
+ settingsViewModel.updateLlmCustomResponsePath(p.config.responsePath)
+ }
+ if (customRequestTemplate.isBlank()) {
+ customRequestTemplate = p.config.requestTemplate
+ settingsViewModel.updateLlmCustomRequestTemplate(p.config.requestTemplate)
+ }
+ if (customMessageFormat.isBlank()) {
+ customMessageFormat = p.config.messageFormat
+ settingsViewModel.updateLlmCustomMessageFormat(p.config.messageFormat)
+ }
+ if (customSystemRole.isBlank()) {
+ customSystemRole = p.config.systemRole ?: "system"
+ settingsViewModel.updateLlmCustomSystemRole(customSystemRole)
+ }
+ if (customUserRole.isBlank()) {
+ customUserRole = p.config.userRole
+ settingsViewModel.updateLlmCustomUserRole(p.config.userRole)
+ }
+ if (customAssistantRole.isBlank()) {
+ customAssistantRole = p.config.assistantRole
+ settingsViewModel.updateLlmCustomAssistantRole(p.config.assistantRole)
+ }
+ }
+ }
+ expanded = false
+ }
+ )
+ }
+ }
+ }
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ val modelsList = fetchedModelsMap[provider] ?: provider.suggestedModels
+
+ Box(modifier = Modifier.fillMaxWidth()) {
+ OutlinedTextField(
+ value = model,
+ onValueChange = {
+ model = it
+ settingsViewModel.updateLlmModel(it)
+ onResetVerificationState()
+ },
+ label = { Text(stringResource(id = R.string.llm_model)) },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth(),
+ supportingText = if (!modelFetchError.isNullOrBlank()) {
+ { Text(modelFetchError, color = MaterialTheme.colorScheme.error) }
+ } else null,
+ trailingIcon = {
+ IconButton(onClick = { modelDropdownExpanded = !modelDropdownExpanded }) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_arrow_drop_down),
+ contentDescription = "Show suggested models"
+ )
+ }
+ }
+ )
+ DropdownMenu(
+ expanded = modelDropdownExpanded,
+ onDismissRequest = { modelDropdownExpanded = false },
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ if (isFetchingModels) {
+ DropdownMenuItem(
+ text = {
+ Row(
+ verticalAlignment = Alignment.CenterVertically,
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ CircularProgressIndicator(
+ modifier = Modifier.size(16.dp),
+ strokeWidth = 2.dp
+ )
+ Spacer(modifier = Modifier.width(12.dp))
+ Text("Fetching models...")
+ }
+ },
+ onClick = {}
+ )
+ } else {
+ if (provider != LlmProvider.CUSTOM) {
+ DropdownMenuItem(
+ text = {
+ Row(
+ verticalAlignment = Alignment.CenterVertically,
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_restart),
+ contentDescription = "Fetch latest",
+ modifier = Modifier.size(16.dp)
+ )
+ Spacer(modifier = Modifier.width(12.dp))
+ Text("Fetch latest models from API")
+ }
+ },
+ onClick = {
+ onFetchModelsForProvider(
+ provider,
+ apiKey2,
+ customUrl,
+ customHeaders
+ )
+ }
+ )
+ if (modelsList.isNotEmpty()) {
+ androidx.compose.material3.HorizontalDivider(
+ modifier = Modifier.padding(vertical = 4.dp)
+ )
+ }
+ }
+
+ if (modelsList.isNotEmpty()) {
+ modelsList.forEach { suggested ->
+ DropdownMenuItem(
+ text = { Text(suggested) },
+ onClick = {
+ model = suggested
+ settingsViewModel.updateLlmModel(suggested)
+ onResetVerificationState()
+ modelDropdownExpanded = false
+ }
+ )
+ }
+ } else if (provider == LlmProvider.CUSTOM) {
+ DropdownMenuItem(
+ text = { Text("No suggested models. Please enter your model name.") },
+ onClick = { modelDropdownExpanded = false }
+ )
+ }
+ }
+ }
+ }
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ OutlinedTextField(
+ value = apiKey2,
+ onValueChange = {
+ apiKey2 = it
+ settingsViewModel.updateChatApiKey(it)
+ onResetVerificationState()
+ },
+ label = { Text(stringResource(id = R.string.chat_api_key)) },
+ visualTransformation = androidx.compose.ui.text.input.PasswordVisualTransformation(),
+ trailingIcon = {
+ IconButton(
+ onClick = {
+ targetKeyField = "chat"
+ showQrScanner = true
+ }
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_qr_code_scanner),
+ contentDescription = "Scan Chat API Key"
+ )
+ }
+ },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+
+ Spacer(modifier = Modifier.height(16.dp))
+
+ // Verification Section
+ VerifySection(
+ verificationState = verificationState,
+ onVerify = {
+ onVerifyModelAndSaveCapabilities(
+ provider,
+ model,
+ apiKey2,
+ customUrl,
+ customHeaders,
+ customResponsePath,
+ customRequestTemplate,
+ customMessageFormat,
+ customSystemRole,
+ customUserRole,
+ customAssistantRole
+ )
+ }
+ )
+ }
+ }
+
+ // CARD 2: CUSTOM PROVIDER DETAILS (ONLY RENDERED IF CUSTOM SELECTED)
+ if (provider == LlmProvider.CUSTOM) {
+ Card(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 8.dp),
+ shape = RoundedCornerShape(16.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.2f)
+ ),
+ elevation = CardDefaults.cardElevation(defaultElevation = 0.dp)
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ SettingsSectionHeader(
+ title = "Custom Endpoint Configuration",
+ description = "Define custom API mappings and endpoints",
+ icon = R.drawable.ic_settings
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+
+ OutlinedTextField(
+ value = customUrl,
+ onValueChange = {
+ customUrl = it
+ settingsViewModel.updateLlmCustomUrl(it)
+ onResetVerificationState()
+ },
+ label = { Text("API Endpoint URL") },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ OutlinedTextField(
+ value = customHeaders,
+ onValueChange = {
+ customHeaders = it
+ settingsViewModel.updateLlmCustomHeaders(it)
+ onResetVerificationState()
+ },
+ label = { Text("Custom Headers (JSON)") },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ OutlinedTextField(
+ value = customResponsePath,
+ onValueChange = {
+ customResponsePath = it
+ settingsViewModel.updateLlmCustomResponsePath(it)
+ onResetVerificationState()
+ },
+ label = { Text("Response Extraction JSON Path") },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ OutlinedTextField(
+ value = customRequestTemplate,
+ onValueChange = {
+ customRequestTemplate = it
+ settingsViewModel.updateLlmCustomRequestTemplate(it)
+ onResetVerificationState()
+ },
+ label = { Text("Request Body JSON Template") },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier
+ .fillMaxWidth()
+ .height(120.dp),
+ maxLines = 10
+ )
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ OutlinedTextField(
+ value = customMessageFormat,
+ onValueChange = {
+ customMessageFormat = it
+ settingsViewModel.updateLlmCustomMessageFormat(it)
+ onResetVerificationState()
+ },
+ label = { Text("Single Message Format (JSON)") },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ OutlinedTextField(
+ value = customSystemRole,
+ onValueChange = {
+ customSystemRole = it
+ settingsViewModel.updateLlmCustomSystemRole(it)
+ onResetVerificationState()
+ },
+ label = { Text("System Role Mapping") },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ OutlinedTextField(
+ value = customUserRole,
+ onValueChange = {
+ customUserRole = it
+ settingsViewModel.updateLlmCustomUserRole(it)
+ onResetVerificationState()
+ },
+ label = { Text("User Role Mapping") },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ OutlinedTextField(
+ value = customAssistantRole,
+ onValueChange = {
+ customAssistantRole = it
+ settingsViewModel.updateLlmCustomAssistantRole(it)
+ onResetVerificationState()
+ },
+ label = { Text("Assistant Role Mapping") },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+ }
+ }
+ }
+
+ // CARD 3: INTEGRATIONS
+ Card(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 8.dp),
+ shape = RoundedCornerShape(16.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.2f)
+ ),
+ elevation = CardDefaults.cardElevation(defaultElevation = 0.dp)
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ SettingsSectionHeader(
+ title = "Integrations",
+ description = "Manage credentials for third-party integrations",
+ icon = R.drawable.ic_settings
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+
+ OutlinedTextField(
+ value = apiKey1,
+ onValueChange = {
+ apiKey1 = it
+ settingsViewModel.updateYoutubeApiKey(it)
+ },
+ label = { Text(stringResource(id = R.string.youtube_api_key)) },
+ visualTransformation = androidx.compose.ui.text.input.PasswordVisualTransformation(),
+ trailingIcon = {
+ IconButton(
+ onClick = {
+ targetKeyField = "youtube"
+ showQrScanner = true
+ }
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_qr_code_scanner),
+ contentDescription = "Scan YouTube API Key"
+ )
+ }
+ },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+ }
+ }
+
+ // CARD 4: SPEECH TO TEXT SETTINGS
+ Card(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 8.dp),
+ shape = RoundedCornerShape(16.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.2f)
+ ),
+ elevation = CardDefaults.cardElevation(defaultElevation = 0.dp)
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ SettingsSectionHeader(
+ title = stringResource(id = R.string.stt_settings_title),
+ description = "Configure how the assistant listens to your speech",
+ icon = R.drawable.ic_mic
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+
+ val isInstalled = isModelInstalled
+ val downloadState = modelDownloadState
+
+ // Option 1: Native STT
+ SelectableOptionCard(
+ selected = sttMode == SttMode.NATIVE,
+ onClick = {
+ sttMode = SttMode.NATIVE
+ settingsViewModel.updateSttMode(SttMode.NATIVE)
+ },
+ title = stringResource(id = R.string.use_native_stt),
+ description = stringResource(id = R.string.native_stt_desc)
+ )
+
+ // Option 2: Parakeet STT (Only enabled if downloaded)
+ SelectableOptionCard(
+ selected = sttMode == SttMode.PARAKEET,
+ enabled = isInstalled,
+ onClick = {
+ sttMode = SttMode.PARAKEET
+ settingsViewModel.updateSttMode(SttMode.PARAKEET)
+ },
+ title = stringResource(id = R.string.use_parakeet_stt),
+ description = if (isInstalled) {
+ stringResource(id = R.string.parakeet_stt_desc_enabled)
+ } else {
+ stringResource(id = R.string.parakeet_stt_desc_disabled)
+ }
+ )
+
+ // Option 3: Hybrid STT (Only enabled if downloaded)
+ SelectableOptionCard(
+ selected = sttMode == SttMode.HYBRID,
+ enabled = isInstalled,
+ onClick = {
+ sttMode = SttMode.HYBRID
+ settingsViewModel.updateSttMode(SttMode.HYBRID)
+ },
+ title = stringResource(id = R.string.use_hybrid_stt),
+ description = if (isInstalled) {
+ stringResource(id = R.string.hybrid_stt_desc_enabled)
+ } else {
+ stringResource(id = R.string.hybrid_stt_desc_disabled)
+ }
+ )
+
+ // Option 4: API STT (Cloud)
+ SelectableOptionCard(
+ selected = sttMode == SttMode.API,
+ onClick = {
+ sttMode = SttMode.API
+ settingsViewModel.updateSttMode(SttMode.API)
+ },
+ title = stringResource(id = R.string.use_api_stt),
+ description = stringResource(id = R.string.api_stt_desc) + " (Configured: ${SpeechConfig.ACTIVE_STT_PROVIDER})"
+ )
+
+ Spacer(modifier = Modifier.height(8.dp))
+
+ // Model Information & Warning Card
+ ModelDownloadCard(
+ title = stringResource(id = R.string.parakeet_model_title),
+ description = stringResource(id = R.string.parakeet_model_desc),
+ isInstalled = isInstalled,
+ downloadState = downloadState,
+ onDownload = onStartModelDownload,
+ onDelete = onDeleteModel,
+ onCancel = onCancelModelDownload,
+ onClear = onDeleteModel,
+ onRetry = onStartModelDownload,
+ installedText = stringResource(id = R.string.status_installed),
+ notDownloadedText = stringResource(id = R.string.status_not_downloaded),
+ deleteButtonText = stringResource(id = R.string.delete_model),
+ downloadButtonText = stringResource(id = R.string.download_model)
+ )
+
+ // Automatically revert selection to NATIVE if model is not installed but current selected is parakeet/hybrid
+ LaunchedEffect(isInstalled) {
+ if (!isInstalled && (sttMode == SttMode.PARAKEET || sttMode == SttMode.HYBRID)) {
+ sttMode = SttMode.NATIVE
+ settingsViewModel.updateSttMode(SttMode.NATIVE)
+ }
+ }
+ }
+ }
+
+ // CARD 5: TEXT TO SPEECH (TTS) SETTINGS
+ Card(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 8.dp),
+ shape = RoundedCornerShape(16.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.2f)
+ ),
+ elevation = CardDefaults.cardElevation(defaultElevation = 0.dp)
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ SettingsSectionHeader(
+ title = "Text-to-Speech Settings",
+ description = "Configure how the assistant speaks back to you",
+ icon = R.drawable.ic_speaker
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+
+ Text(
+ text = "Using an offline TTS model will require device storage for voice processing.",
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.8f)
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+
+ // Option 1: Native TTS
+ SelectableOptionCard(
+ selected = ttsMode == com.app.assistant.tts.TtsMode.NATIVE,
+ onClick = {
+ ttsMode = com.app.assistant.tts.TtsMode.NATIVE
+ settingsViewModel.updateTtsMode(com.app.assistant.tts.TtsMode.NATIVE)
+ },
+ title = "Use Native TTS (Fast)",
+ description = "Uses the built-in system text-to-speech engine.",
+ trailingContent = {
+ IconButton(
+ onClick = {
+ try {
+ val intent = android.content.Intent("com.android.settings.TTS_SETTINGS")
+ context.startActivity(intent)
+ } catch (e: Exception) {
+ android.widget.Toast.makeText(
+ context,
+ "Could not open TTS settings",
+ android.widget.Toast.LENGTH_SHORT
+ ).show()
+ }
+ },
+ modifier = Modifier.size(36.dp)
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_settings),
+ contentDescription = "Open System TTS Settings",
+ tint = MaterialTheme.colorScheme.primary,
+ modifier = Modifier.size(20.dp)
+ )
+ }
+ }
+ )
+
+ // Option 2: Offline TTS
+ SelectableOptionCard(
+ selected = ttsMode == com.app.assistant.tts.TtsMode.OFFLINE,
+ enabled = isTtsInstalled,
+ onClick = {
+ ttsMode = com.app.assistant.tts.TtsMode.OFFLINE
+ settingsViewModel.updateTtsMode(com.app.assistant.tts.TtsMode.OFFLINE)
+ },
+ title = "Use Offline TTS Model",
+ description = if (isTtsInstalled) {
+ "Uses Supertonic/ONNX engine on-device without internet."
+ } else {
+ "Uses Supertonic/ONNX engine on-device without internet. Requires downloading the model first."
+ }
+ )
+
+ // Option 3: API TTS (Cloud)
+ SelectableOptionCard(
+ selected = ttsMode == com.app.assistant.tts.TtsMode.API,
+ onClick = {
+ ttsMode = com.app.assistant.tts.TtsMode.API
+ settingsViewModel.updateTtsMode(com.app.assistant.tts.TtsMode.API)
+ },
+ title = stringResource(id = R.string.use_api_tts),
+ description = stringResource(id = R.string.api_tts_desc) + " (Configured: ${SpeechConfig.ACTIVE_TTS_PROVIDER})"
+ )
+
+ Spacer(modifier = Modifier.height(8.dp))
+
+ OutlinedTextField(
+ value = edgeTtsKey,
+ onValueChange = {
+ edgeTtsKey = it
+ settingsViewModel.updateEdgeTtsSubscriptionKey(it)
+ },
+ label = { Text(stringResource(id = R.string.edge_tts_subscription_key)) },
+ visualTransformation = androidx.compose.ui.text.input.PasswordVisualTransformation(),
+ trailingIcon = {
+ IconButton(
+ onClick = {
+ targetKeyField = "edge_tts"
+ showQrScanner = true
+ }
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_qr_code_scanner),
+ contentDescription = "Scan Edge TTS Key"
+ )
+ }
+ },
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ )
+
+ Spacer(modifier = Modifier.height(12.dp))
+
+ ModelDownloadCard(
+ title = "Supertonic English Model (int8)",
+ description = "Supertonic v3 English int8 model.",
+ isInstalled = isTtsInstalled,
+ downloadState = ttsDownloadState,
+ onDownload = onStartTtsModelDownload,
+ onDelete = onDeleteTtsModel,
+ onCancel = onCancelTtsModelDownload,
+ onClear = onDeleteTtsModel,
+ onRetry = onStartTtsModelDownload,
+ installedText = "Installed (~66 MB)",
+ notDownloadedText = "Status: Not Downloaded",
+ deleteButtonText = "Delete",
+ downloadButtonText = "Download"
+ )
+ }
+ }
+
+ //Hiding for now due to complexity of response stream its difficult to implement offline translation, but it will be added in future for sure
+ // CARD 6: OFFLINE TRANSLATION SETTINGS
+// Card(
+// modifier = Modifier
+// .fillMaxWidth()
+// .padding(vertical = 8.dp),
+// shape = RoundedCornerShape(16.dp),
+// border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+// colors = CardDefaults.cardColors(
+// containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.2f),
+// ),
+// elevation = CardDefaults.cardElevation(defaultElevation = 0.dp)
+// ) {
+// Column(modifier = Modifier.padding(16.dp)) {
+// SettingsSectionHeader(
+// title = "Offline Translation Settings",
+// description = "Translate queries and responses offline using Google ML Kit models.",
+// icon = R.drawable.ic_translate
+// )
+// Spacer(modifier = Modifier.height(8.dp))
+//
+// Row(
+// modifier = Modifier
+// .fillMaxWidth()
+// .padding(vertical = 8.dp),
+// verticalAlignment = Alignment.CenterVertically
+// ) {
+// Column(modifier = Modifier.weight(1f)) {
+// Text(
+// text = "Enable Translation",
+// style = MaterialTheme.typography.bodyLarge,
+// fontWeight = FontWeight.Bold
+// )
+// Text(
+// text = "Translate chat messages and voice input",
+// style = MaterialTheme.typography.bodySmall,
+// color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.8f)
+// )
+// }
+// Switch(
+// checked = isTranslationEnabled,
+// onCheckedChange = onTranslationEnabledChange
+// )
+// }
+//
+// if (isTranslationEnabled) {
+// androidx.compose.material3.HorizontalDivider(modifier = Modifier.padding(vertical = 8.dp))
+//
+// val activeLangName = remember(activeLanguageCode, translationLanguages) {
+// translationLanguages.find { it.second == activeLanguageCode }?.first?.let { formatLanguageName(it) } ?: activeLanguageCode.uppercase()
+// }
+//
+// Card(
+// onClick = { showTranslationSheet = true },
+// modifier = Modifier
+// .fillMaxWidth()
+// .padding(vertical = 6.dp),
+// shape = RoundedCornerShape(12.dp),
+// border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+// colors = CardDefaults.cardColors(
+// containerColor = MaterialTheme.colorScheme.surface
+// )
+// ) {
+// Row(
+// modifier = Modifier
+// .fillMaxWidth()
+// .padding(14.dp),
+// verticalAlignment = Alignment.CenterVertically
+// ) {
+// Icon(
+// painter = painterResource(id = R.drawable.ic_translate),
+// contentDescription = null,
+// tint = MaterialTheme.colorScheme.primary,
+// modifier = Modifier.size(24.dp)
+// )
+// Spacer(modifier = Modifier.width(16.dp))
+// Column(modifier = Modifier.weight(1f)) {
+// Text(
+// text = "Default Language",
+// style = MaterialTheme.typography.bodyLarge,
+// fontWeight = FontWeight.Bold,
+// color = MaterialTheme.colorScheme.onSurface
+// )
+// Spacer(modifier = Modifier.height(2.dp))
+// Text(
+// text = "Selected: $activeLangName",
+// style = MaterialTheme.typography.bodySmall,
+// color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.8f)
+// )
+// }
+// Icon(
+// painter = painterResource(id = R.drawable.ic_arrow_drop_down),
+// contentDescription = "Select language",
+// tint = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.8f),
+// modifier = Modifier.size(24.dp)
+// )
+// }
+// }
+//
+// Spacer(modifier = Modifier.height(8.dp))
+//
+// Button(
+// onClick = { showTranslationSheet = true },
+// shape = RoundedCornerShape(8.dp),
+// modifier = Modifier.fillMaxWidth().height(40.dp)
+// ) {
+// Text("Manage Translation Models", style = MaterialTheme.typography.labelMedium)
+// }
+// }
+// }
+// }
+
+ // CARD 7: DEFAULT ASSISTANT SETTINGS
+ Card(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 8.dp),
+ shape = RoundedCornerShape(16.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.2f)
+ ),
+ elevation = CardDefaults.cardElevation(defaultElevation = 0.dp)
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ SettingsSectionHeader(
+ title = stringResource(id = R.string.default_assistant_title),
+ description = stringResource(id = R.string.default_assistant_desc),
+ icon = R.drawable.ic_settings
+ )
+ Spacer(modifier = Modifier.height(12.dp))
+
+ Row(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 8.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ Column(modifier = Modifier.weight(1f)) {
+ Text(
+ text = if (isDefaultAssistant) {
+ stringResource(id = R.string.default_assistant_status_active)
+ } else {
+ stringResource(id = R.string.default_assistant_status_inactive)
+ },
+ style = MaterialTheme.typography.bodyLarge,
+ fontWeight = FontWeight.Bold,
+ color = if (isDefaultAssistant) {
+ MaterialTheme.colorScheme.primary
+ } else {
+ MaterialTheme.colorScheme.error
+ }
+ )
+ }
+ if (isDefaultAssistant) {
+ Icon(
+ imageVector = Icons.Default.Check,
+ contentDescription = "Active",
+ tint = MaterialTheme.colorScheme.primary,
+ modifier = Modifier.size(24.dp)
+ )
+ } else {
+ Icon(
+ imageVector = Icons.Default.Close,
+ contentDescription = "Inactive",
+ tint = MaterialTheme.colorScheme.error,
+ modifier = Modifier.size(24.dp)
+ )
+ }
+ }
+
+ Spacer(modifier = Modifier.height(8.dp))
+
+ Button(
+ onClick = {
+ openAssistantSettings(context)
+ },
+ shape = RoundedCornerShape(8.dp),
+ modifier = Modifier
+ .fillMaxWidth()
+ .height(40.dp)
+ ) {
+ Text(
+ text = if (isDefaultAssistant) {
+ stringResource(id = R.string.default_assistant_btn_open_settings)
+ } else {
+ stringResource(id = R.string.default_assistant_btn_set)
+ },
+ style = MaterialTheme.typography.labelMedium
+ )
+ }
+ }
+ }
+
+ Spacer(modifier = Modifier.height(24.dp))
+ Box(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(bottom = 24.dp),
+ contentAlignment = Alignment.Center
+ ) {
+ Text(
+ text = stringResource(id = R.string.app_version, versionName),
+ style = MaterialTheme.typography.labelMedium,
+ fontWeight = FontWeight.Medium,
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.6f)
+ )
+ }
+ }
+ }
+
+ if (showTranslationSheet) {
+ val sheetState = rememberModalBottomSheetState()
+ TranslationLanguageManagerBottomSheet(
+ sheetState = sheetState,
+ languages = translationLanguages,
+ activeLanguageCode = activeLanguageCode,
+ downloadedLanguages = downloadedTranslationLanguages,
+ downloadingLanguages = downloadingTranslationLanguages,
+ onLanguageSelected = {
+ onLanguageSelected(it)
+ },
+ onDownloadLanguage = {
+ onDownloadTranslationModel(it)
+ },
+ onDeleteLanguage = {
+ onDeleteTranslationModel(it)
+ },
+ onDismissRequest = { showTranslationSheet = false }
+ )
+ }
+
+ if (showQrScanner) {
+ QrScannerDialog(
+ onDismissRequest = {
+ showQrScanner = false
+ targetKeyField = null
+ },
+ onCodeScanned = { rawCode ->
+ showQrScanner = false
+ val trimmed = rawCode.trim()
+ try {
+ if (trimmed.startsWith("{") && trimmed.endsWith("}")) {
+ val jsonObject = org.json.JSONObject(trimmed)
+ val importedKeys = mutableListOf()
+
+ val chatKeys = listOf("chat_api_key", "chatApiKey", "gemini_api_key", "groq_api_key", "api_key", "apiKey", "gemini_key", "groq_key")
+ for (key in chatKeys) {
+ if (jsonObject.has(key)) {
+ val value = jsonObject.getString(key)
+ if (value.isNotBlank()) {
+ apiKey2 = value
+ settingsViewModel.updateChatApiKey(value)
+ onResetVerificationState()
+ importedKeys.add("Chat API Key")
+ }
+ break
+ }
+ }
+
+ val youtubeKeys = listOf("youtube_api_key", "youtubeApiKey", "youtube_key", "youtubeKey")
+ for (key in youtubeKeys) {
+ if (jsonObject.has(key)) {
+ val value = jsonObject.getString(key)
+ if (value.isNotBlank()) {
+ apiKey1 = value
+ settingsViewModel.updateYoutubeApiKey(value)
+ importedKeys.add("YouTube API Key")
+ }
+ break
+ }
+ }
+
+ val ttsKeys = listOf("edge_tts_key", "edgeTtsKey", "edge_tts_subscription_key", "tts_key", "ttsKey")
+ for (key in ttsKeys) {
+ if (jsonObject.has(key)) {
+ val value = jsonObject.getString(key)
+ if (value.isNotBlank()) {
+ edgeTtsKey = value
+ settingsViewModel.updateEdgeTtsSubscriptionKey(value)
+ importedKeys.add("Edge TTS Key")
+ }
+ break
+ }
+ }
+
+ if (importedKeys.isNotEmpty()) {
+ android.widget.Toast.makeText(
+ context,
+ "Successfully imported: ${importedKeys.joinToString(", ")}",
+ android.widget.Toast.LENGTH_LONG
+ ).show()
+ } else {
+ android.widget.Toast.makeText(
+ context,
+ "QR code scanned but no matching API keys found in JSON",
+ android.widget.Toast.LENGTH_LONG
+ ).show()
+ }
+ } else {
+ if (trimmed.isNotBlank()) {
+ when (targetKeyField) {
+ "chat" -> {
+ apiKey2 = trimmed
+ settingsViewModel.updateChatApiKey(trimmed)
+ onResetVerificationState()
+ android.widget.Toast.makeText(context, "Imported Chat API Key", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ "youtube" -> {
+ apiKey1 = trimmed
+ settingsViewModel.updateYoutubeApiKey(trimmed)
+ android.widget.Toast.makeText(context, "Imported YouTube API Key", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ "edge_tts" -> {
+ edgeTtsKey = trimmed
+ settingsViewModel.updateEdgeTtsSubscriptionKey(trimmed)
+ android.widget.Toast.makeText(context, "Imported Edge TTS Key", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ }
+ }
+ }
+ } catch (e: Exception) {
+ if (trimmed.isNotBlank()) {
+ when (targetKeyField) {
+ "chat" -> {
+ apiKey2 = trimmed
+ settingsViewModel.updateChatApiKey(trimmed)
+ onResetVerificationState()
+ android.widget.Toast.makeText(context, "Imported Chat API Key", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ "youtube" -> {
+ apiKey1 = trimmed
+ settingsViewModel.updateYoutubeApiKey(trimmed)
+ android.widget.Toast.makeText(context, "Imported YouTube API Key", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ "edge_tts" -> {
+ edgeTtsKey = trimmed
+ settingsViewModel.updateEdgeTtsSubscriptionKey(trimmed)
+ android.widget.Toast.makeText(context, "Imported Edge TTS Key", android.widget.Toast.LENGTH_SHORT).show()
+ }
+ }
+ }
+ }
+ targetKeyField = null
+ }
+ )
+ }
+}
+
+@Composable
+fun VerifySection(
+ verificationState: VerificationState,
+ onVerify: () -> Unit
+) {
+ Column(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(top = 8.dp)
+ ) {
+ when (verificationState) {
+ is VerificationState.Idle -> {
+ Button(
+ onClick = onVerify,
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ Text("Verify & Detect Capabilities", fontWeight = FontWeight.Bold)
+ }
+ }
+ is VerificationState.Verifying -> {
+ Card(
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.primaryContainer.copy(alpha = 0.2f)
+ ),
+ shape = RoundedCornerShape(12.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.primary.copy(alpha = 0.3f)),
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ Row(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(16.dp),
+ verticalAlignment = Alignment.CenterVertically,
+ horizontalArrangement = Arrangement.Center
+ ) {
+ CircularProgressIndicator(
+ modifier = Modifier.size(20.dp),
+ strokeWidth = 2.5.dp
+ )
+ Spacer(modifier = Modifier.width(12.dp))
+ Text(
+ text = "Verifying connection and capabilities...",
+ style = MaterialTheme.typography.bodyMedium,
+ fontWeight = FontWeight.Medium,
+ color = MaterialTheme.colorScheme.onSurface
+ )
+ }
+ }
+ }
+ is VerificationState.Success -> {
+ Card(
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.primaryContainer.copy(alpha = 0.1f)
+ ),
+ shape = RoundedCornerShape(12.dp),
+ border = BorderStroke(1.dp, Color(0xFF4CAF50).copy(alpha = 0.4f)),
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ Row(
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ Icon(
+ imageVector = Icons.Default.Check,
+ contentDescription = "Success",
+ tint = Color(0xFF4CAF50),
+ modifier = Modifier.size(20.dp)
+ )
+ Spacer(modifier = Modifier.width(8.dp))
+ Text(
+ text = "Verification Successful!",
+ color = Color(0xFF4CAF50),
+ fontWeight = FontWeight.Bold,
+ style = MaterialTheme.typography.bodyMedium
+ )
+ }
+ Spacer(modifier = Modifier.height(12.dp))
+ Text(
+ text = "Detected Model Capabilities:",
+ style = MaterialTheme.typography.bodySmall,
+ fontWeight = FontWeight.SemiBold,
+ color = MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+
+ Column(
+ verticalArrangement = Arrangement.spacedBy(4.dp)
+ ) {
+ CapabilityRow("Text Completion Support", true)
+ CapabilityRow("Vision / Image Support", verificationState.capabilities.hasImageInput)
+ CapabilityRow("Audio Support", verificationState.capabilities.hasAudioInput)
+ CapabilityRow("Video Support", verificationState.capabilities.hasVideoInput)
+ CapabilityRow("Document Support (PDF/TXT)", verificationState.capabilities.hasDocumentInput)
+ }
+
+ Spacer(modifier = Modifier.height(16.dp))
+ OutlinedButton(
+ onClick = onVerify,
+ shape = RoundedCornerShape(12.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.outline.copy(alpha = 0.5f)),
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ Text("Re-Verify Settings", fontWeight = FontWeight.Bold)
+ }
+ }
+ }
+ }
+ is VerificationState.Error -> {
+ Card(
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.errorContainer.copy(alpha = 0.15f)
+ ),
+ shape = RoundedCornerShape(12.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.error.copy(alpha = 0.4f)),
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ Row(
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ Icon(
+ imageVector = Icons.Default.Close,
+ contentDescription = "Failed",
+ tint = MaterialTheme.colorScheme.error,
+ modifier = Modifier.size(20.dp)
+ )
+ Spacer(modifier = Modifier.width(8.dp))
+ Text(
+ text = "Verification Failed",
+ color = MaterialTheme.colorScheme.error,
+ fontWeight = FontWeight.Bold,
+ style = MaterialTheme.typography.bodyMedium
+ )
+ }
+ Spacer(modifier = Modifier.height(8.dp))
+ Text(
+ text = verificationState.message,
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onErrorContainer
+ )
+ Spacer(modifier = Modifier.height(16.dp))
+ Button(
+ onClick = onVerify,
+ colors = ButtonDefaults.buttonColors(containerColor = MaterialTheme.colorScheme.error),
+ shape = RoundedCornerShape(12.dp),
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ Text("Retry Verification", fontWeight = FontWeight.Bold)
+ }
+ }
+ }
+ }
+ }
+ }
+}
+
+@Composable
+fun CapabilityRow(label: String, supported: Boolean) {
+ val isDark = isSystemInDarkTheme()
+ val tintColor = if (supported) {
+ if (isDark) Color(0xFF81C784) else Color(0xFF2E7D32)
+ } else {
+ MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.35f)
+ }
+
+ Row(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 4.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ Icon(
+ imageVector = if (supported) Icons.Default.Check else Icons.Default.Close,
+ contentDescription = if (supported) "Supported" else "Unsupported",
+ tint = tintColor,
+ modifier = Modifier.size(16.dp)
+ )
+ Spacer(modifier = Modifier.width(10.dp))
+ Text(
+ text = label,
+ style = MaterialTheme.typography.bodyMedium,
+ color = if (supported) MaterialTheme.colorScheme.onSurface else MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.6f)
+ )
+ }
+}
+
+@Composable
+fun SettingsSectionHeader(
+ title: String,
+ description: String? = null,
+ icon: Int? = null,
+ modifier: Modifier = Modifier
+) {
+ Column(
+ modifier = modifier
+ .fillMaxWidth()
+ .padding(bottom = 12.dp)
+ ) {
+ Row(
+ verticalAlignment = Alignment.CenterVertically,
+ modifier = Modifier.fillMaxWidth()
+ ) {
+ if (icon != null) {
+ Icon(
+ painter = painterResource(id = icon),
+ contentDescription = null,
+ tint = MaterialTheme.colorScheme.primary,
+ modifier = Modifier.size(22.dp)
+ )
+ Spacer(modifier = Modifier.width(10.dp))
+ }
+ Text(
+ text = title,
+ style = MaterialTheme.typography.titleMedium,
+ fontWeight = FontWeight.Bold,
+ color = MaterialTheme.colorScheme.primary
+ )
+ }
+ if (description != null) {
+ Spacer(modifier = Modifier.height(2.dp))
+ Text(
+ text = description,
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.8f),
+ modifier = Modifier.padding(start = if (icon != null) 32.dp else 0.dp)
+ )
+ }
+ }
+}
+
+@Composable
+fun StatusBadge(
+ text: String,
+ isInstalled: Boolean,
+ modifier: Modifier = Modifier
+) {
+ val isDark = isSystemInDarkTheme()
+ val containerColor = if (isInstalled) {
+ if (isDark) Color(0xFF1B5E20).copy(alpha = 0.25f) else Color(0xFFE8F5E9)
+ } else {
+ MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.6f)
+ }
+ val contentColor = if (isInstalled) {
+ if (isDark) Color(0xFF81C784) else Color(0xFF2E7D32)
+ } else {
+ MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.8f)
+ }
+ val borderColor = if (isInstalled) {
+ if (isDark) Color(0xFF81C784).copy(alpha = 0.3f) else Color(0xFFC8E6C9)
+ } else {
+ MaterialTheme.colorScheme.outline.copy(alpha = 0.2f)
+ }
+
+ Surface(
+ shape = RoundedCornerShape(8.dp),
+ border = BorderStroke(1.dp, borderColor),
+ color = containerColor,
+ modifier = modifier
+ ) {
+ Row(
+ modifier = Modifier.padding(horizontal = 8.dp, vertical = 4.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ if (isInstalled) {
+ Icon(
+ imageVector = Icons.Default.Check,
+ contentDescription = null,
+ tint = contentColor,
+ modifier = Modifier.size(12.dp)
+ )
+ Spacer(modifier = Modifier.width(4.dp))
+ }
+ Text(
+ text = text,
+ style = MaterialTheme.typography.labelSmall,
+ fontWeight = FontWeight.Bold,
+ color = contentColor
+ )
+ }
+ }
+}
+
+@Composable
+fun SelectableOptionCard(
+ selected: Boolean,
+ enabled: Boolean = true,
+ onClick: () -> Unit,
+ title: String,
+ description: String,
+ trailingContent: (@Composable () -> Unit)? = null,
+ modifier: Modifier = Modifier
+) {
+ val isDark = isSystemInDarkTheme()
+ val alpha = if (enabled) 1.0f else 0.5f
+
+ val borderStroke = if (selected && enabled) {
+ BorderStroke(2.dp, MaterialTheme.colorScheme.primary)
+ } else {
+ BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f))
+ }
+
+ val containerColor = if (selected && enabled) {
+ MaterialTheme.colorScheme.primaryContainer.copy(alpha = if (isDark) 0.15f else 0.08f)
+ } else {
+ MaterialTheme.colorScheme.surface
+ }
+
+ Surface(
+ shape = RoundedCornerShape(12.dp),
+ border = borderStroke,
+ color = containerColor,
+ modifier = modifier
+ .fillMaxWidth()
+ .padding(vertical = 6.dp)
+ .clickable(enabled = enabled) { onClick() }
+ ) {
+ Row(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(14.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ RadioButton(
+ selected = selected,
+ enabled = enabled,
+ onClick = if (enabled) onClick else null
+ )
+ Spacer(modifier = Modifier.width(12.dp))
+ Column(modifier = Modifier.weight(1f)) {
+ Text(
+ text = title,
+ style = MaterialTheme.typography.bodyLarge,
+ fontWeight = FontWeight.Bold,
+ color = MaterialTheme.colorScheme.onSurface.copy(alpha = alpha)
+ )
+ if (description.isNotEmpty()) {
+ Spacer(modifier = Modifier.height(2.dp))
+ Text(
+ text = description,
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = alpha * 0.8f)
+ )
+ }
+ }
+ if (trailingContent != null) {
+ Spacer(modifier = Modifier.width(8.dp))
+ trailingContent()
+ }
+ }
+ }
+}
+
+@Composable
+fun ModelDownloadCard(
+ title: String,
+ description: String,
+ isInstalled: Boolean,
+ downloadState: DownloadState,
+ onDownload: () -> Unit,
+ onDelete: () -> Unit,
+ onCancel: () -> Unit,
+ onClear: () -> Unit,
+ onRetry: () -> Unit,
+ installedText: String,
+ notDownloadedText: String,
+ deleteButtonText: String,
+ downloadButtonText: String,
+ modifier: Modifier = Modifier
+) {
+ Card(
+ colors = CardDefaults.cardColors(
+ containerColor = MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.25f)
+ ),
+ shape = RoundedCornerShape(12.dp),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant.copy(alpha = 0.4f)),
+ modifier = modifier
+ .fillMaxWidth()
+ .padding(bottom = 12.dp)
+ ) {
+ Column(modifier = Modifier.padding(16.dp)) {
+ Text(
+ text = title,
+ style = MaterialTheme.typography.titleSmall,
+ fontWeight = FontWeight.Bold,
+ color = MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ Spacer(modifier = Modifier.height(4.dp))
+ Text(
+ text = description,
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.8f)
+ )
+
+ Spacer(modifier = Modifier.height(16.dp))
+
+ when (downloadState) {
+ is DownloadState.Idle -> {
+ Row(
+ modifier = Modifier.fillMaxWidth(),
+ horizontalArrangement = Arrangement.SpaceBetween,
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ StatusBadge(
+ text = if (isInstalled) installedText else notDownloadedText,
+ isInstalled = isInstalled
+ )
+ if (isInstalled) {
+ OutlinedButton(
+ onClick = onDelete,
+ shape = RoundedCornerShape(8.dp),
+ colors = ButtonDefaults.outlinedButtonColors(
+ contentColor = MaterialTheme.colorScheme.error
+ ),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.error.copy(alpha = 0.5f)),
+ contentPadding = androidx.compose.foundation.layout.PaddingValues(horizontal = 12.dp, vertical = 6.dp),
+ modifier = Modifier.height(36.dp)
+ ) {
+ Icon(
+ imageVector = Icons.Default.Delete,
+ contentDescription = null,
+ modifier = Modifier.size(16.dp)
+ )
+ Spacer(modifier = Modifier.width(6.dp))
+ Text(deleteButtonText, style = MaterialTheme.typography.labelMedium)
+ }
+ } else {
+ Button(
+ onClick = onDownload,
+ shape = RoundedCornerShape(8.dp),
+ contentPadding = androidx.compose.foundation.layout.PaddingValues(horizontal = 16.dp, vertical = 6.dp),
+ modifier = Modifier.height(36.dp)
+ ) {
+ Text(downloadButtonText, style = MaterialTheme.typography.labelMedium)
+ }
+ }
+ }
+ }
+ is DownloadState.Downloading -> {
+ Column(modifier = Modifier.fillMaxWidth()) {
+ Row(
+ modifier = Modifier.fillMaxWidth(),
+ horizontalArrangement = Arrangement.SpaceBetween,
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ val progressPercent = (downloadState.progress * 100).toInt()
+ val downloadedMb = (downloadState.currentBytes.toDouble() / (1024 * 1024)).toInt()
+ val totalMb = (downloadState.totalBytes.toDouble() / (1024 * 1024)).toInt()
+ val progressText = if (totalMb > 0) {
+ "Downloading… $progressPercent% ($downloadedMb MB / $totalMb MB)"
+ } else {
+ "Downloading…"
+ }
+ Text(
+ text = progressText,
+ style = MaterialTheme.typography.bodySmall,
+ fontWeight = FontWeight.Bold,
+ color = MaterialTheme.colorScheme.primary
+ )
+ IconButton(
+ onClick = onCancel,
+ modifier = Modifier.size(24.dp)
+ ) {
+ Icon(
+ imageVector = Icons.Default.Close,
+ contentDescription = "Cancel",
+ tint = MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ }
+ }
+ Spacer(modifier = Modifier.height(8.dp))
+ androidx.compose.material3.LinearProgressIndicator(
+ progress = { downloadState.progress },
+ modifier = Modifier.fillMaxWidth(),
+ strokeCap = androidx.compose.ui.graphics.StrokeCap.Round
+ )
+ }
+ }
+ is DownloadState.Completed -> {
+ Row(
+ modifier = Modifier.fillMaxWidth(),
+ horizontalArrangement = Arrangement.SpaceBetween,
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ StatusBadge(
+ text = "Download Completed!",
+ isInstalled = true
+ )
+ OutlinedButton(
+ onClick = onDelete,
+ shape = RoundedCornerShape(8.dp),
+ colors = ButtonDefaults.outlinedButtonColors(
+ contentColor = MaterialTheme.colorScheme.error
+ ),
+ border = BorderStroke(1.dp, MaterialTheme.colorScheme.error.copy(alpha = 0.5f)),
+ contentPadding = androidx.compose.foundation.layout.PaddingValues(horizontal = 12.dp, vertical = 6.dp),
+ modifier = Modifier.height(36.dp)
+ ) {
+ Icon(
+ imageVector = Icons.Default.Delete,
+ contentDescription = null,
+ modifier = Modifier.size(16.dp)
+ )
+ Spacer(modifier = Modifier.width(6.dp))
+ Text(deleteButtonText, style = MaterialTheme.typography.labelMedium)
+ }
+ }
+ }
+ is DownloadState.Error -> {
+ Column(modifier = Modifier.fillMaxWidth()) {
+ Text(
+ text = "Download Failed: ${downloadState.message}",
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.error,
+ fontWeight = FontWeight.SemiBold
+ )
+ Spacer(modifier = Modifier.height(8.dp))
+ Row(
+ modifier = Modifier.fillMaxWidth(),
+ horizontalArrangement = Arrangement.End,
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ OutlinedButton(
+ onClick = onClear,
+ shape = RoundedCornerShape(8.dp),
+ contentPadding = androidx.compose.foundation.layout.PaddingValues(horizontal = 12.dp, vertical = 6.dp),
+ modifier = Modifier.height(36.dp)
+ ) {
+ Text("Clear", style = MaterialTheme.typography.labelMedium)
+ }
+ Spacer(modifier = Modifier.width(8.dp))
+ Button(
+ onClick = onRetry,
+ shape = RoundedCornerShape(8.dp),
+ contentPadding = androidx.compose.foundation.layout.PaddingValues(horizontal = 12.dp, vertical = 6.dp),
+ modifier = Modifier.height(36.dp)
+ ) {
+ Text("Retry", style = MaterialTheme.typography.labelMedium)
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun SettingsScreenContentPreview() {
+ AssistantTheme {
+ val context = LocalContext.current
+ val dummyViewModel = remember {
+ SettingsViewModel(
+ settingsRepository = com.app.assistant.repository.SettingsRepository(context),
+ okHttpClient = okhttp3.OkHttpClient()
+ )
+ }
+ SettingsScreenContent(
+ settingsViewModel = dummyViewModel,
+ initialYoutubeKey = "AIzaSy...",
+ initialEdgeTtsKey = "EdgeKey...",
+ initialChatKey = "gsk_...",
+ initialProvider = LlmProvider.GROQ,
+ initialModel = LlmProvider.GROQ.defaultModel,
+ initialCustomUrl = "",
+ initialCustomHeaders = "",
+ initialCustomResponsePath = "",
+ initialCustomRequestTemplate = "",
+ initialCustomMessageFormat = "",
+ initialCustomSystemRole = "system",
+ initialCustomUserRole = "user",
+ initialCustomAssistantRole = "assistant",
+ initialSttMode = SttMode.NATIVE,
+ initialTtsMode = com.app.assistant.tts.TtsMode.NATIVE,
+ isTtsInstalled = false,
+ verificationState = VerificationState.Idle,
+ fetchedModelsMap = emptyMap(),
+ isFetchingModels = false,
+ modelFetchError = null,
+ isModelInstalled = false,
+ modelDownloadState = DownloadState.Idle,
+ ttsDownloadState = DownloadState.Idle,
+ isTranslationEnabled = false,
+ activeLanguageCode = "en",
+ downloadedTranslationLanguages = emptySet(),
+ downloadingTranslationLanguages = emptySet(),
+ translationLanguages = emptyList(),
+ isDefaultAssistant = false,
+ onBack = {},
+ onResetVerificationState = {},
+ onFetchModelsForProvider = { _, _, _, _ -> },
+ onVerifyModelAndSaveCapabilities = { _, _, _, _, _, _, _, _, _, _, _ -> },
+ onStartModelDownload = {},
+ onDeleteModel = {},
+ onCancelModelDownload = {},
+ onStartTtsModelDownload = {},
+ onDeleteTtsModel = {},
+ onCancelTtsModelDownload = {},
+ onTranslationEnabledChange = {},
+ onLanguageSelected = {},
+ onDownloadTranslationModel = {},
+ onDeleteTranslationModel = {}
+ )
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun SettingsScreenContentCustomPreview() {
+ AssistantTheme {
+ val context = LocalContext.current
+ val dummyViewModel = remember {
+ SettingsViewModel(
+ settingsRepository = com.app.assistant.repository.SettingsRepository(context),
+ okHttpClient = okhttp3.OkHttpClient()
+ )
+ }
+ SettingsScreenContent(
+ settingsViewModel = dummyViewModel,
+ initialYoutubeKey = "",
+ initialEdgeTtsKey = "",
+ initialChatKey = "",
+ initialProvider = LlmProvider.CUSTOM,
+ initialModel = "my-custom-model",
+ initialCustomUrl = "https://my-custom-endpoint.com/v1",
+ initialCustomHeaders = "{}",
+ initialCustomResponsePath = "$.choices[0].text",
+ initialCustomRequestTemplate = "{}",
+ initialCustomMessageFormat = "{}",
+ initialCustomSystemRole = "system",
+ initialCustomUserRole = "user",
+ initialCustomAssistantRole = "assistant",
+ initialSttMode = SttMode.PARAKEET,
+ initialTtsMode = com.app.assistant.tts.TtsMode.OFFLINE,
+ isTtsInstalled = true,
+ verificationState = VerificationState.Success(
+ capabilities = com.app.assistant.llm.ModelCapabilities(
+ hasImageInput = true,
+ hasAudioInput = false,
+ hasVideoInput = false,
+ hasDocumentInput = true
+ )
+ ),
+ fetchedModelsMap = mapOf(LlmProvider.CUSTOM to listOf("my-custom-model", "another-one")),
+ isFetchingModels = false,
+ modelFetchError = null,
+ isModelInstalled = true,
+ modelDownloadState = DownloadState.Idle,
+ ttsDownloadState = DownloadState.Idle,
+ isTranslationEnabled = false,
+ activeLanguageCode = "en",
+ downloadedTranslationLanguages = emptySet(),
+ downloadingTranslationLanguages = emptySet(),
+ translationLanguages = emptyList(),
+ isDefaultAssistant = true,
+ onBack = {},
+ onResetVerificationState = {},
+ onFetchModelsForProvider = { _, _, _, _ -> },
+ onVerifyModelAndSaveCapabilities = { _, _, _, _, _, _, _, _, _, _, _ -> },
+ onStartModelDownload = {},
+ onDeleteModel = {},
+ onCancelModelDownload = {},
+ onStartTtsModelDownload = {},
+ onDeleteTtsModel = {},
+ onCancelTtsModelDownload = {},
+ onTranslationEnabledChange = {},
+ onLanguageSelected = {},
+ onDownloadTranslationModel = {},
+ onDeleteTranslationModel = {}
+ )
+ }
+}
+
+@OptIn(ExperimentalMaterial3Api::class)
+@Composable
+fun TranslationLanguageManagerBottomSheet(
+ sheetState: SheetState,
+ languages: List>,
+ activeLanguageCode: String,
+ downloadedLanguages: Set,
+ downloadingLanguages: Set,
+ onLanguageSelected: (String) -> Unit,
+ onDownloadLanguage: (String) -> Unit,
+ onDeleteLanguage: (String) -> Unit,
+ onDismissRequest: () -> Unit,
+) {
+ var searchQuery by remember { mutableStateOf("") }
+ val filteredLanguages = remember(searchQuery, languages) {
+ languages.filter { it.first.contains(searchQuery, true) }
+ }
+
+ ModalBottomSheet(
+ onDismissRequest = onDismissRequest,
+ sheetState = sheetState,
+ ) {
+ Column(
+ modifier = Modifier
+ .fillMaxSize()
+ .padding(bottom = 16.dp)
+ ) {
+ Text(
+ text = "Translation Language Manager",
+ style = MaterialTheme.typography.titleMedium,
+ fontWeight = FontWeight.Bold,
+ modifier = Modifier.padding(horizontal = 16.dp, vertical = 8.dp)
+ )
+
+ OutlinedTextField(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(horizontal = 16.dp, vertical = 8.dp),
+ value = searchQuery,
+ placeholder = { Text("Search languages…") },
+ singleLine = true,
+ onValueChange = { searchQuery = it },
+ shape = RoundedCornerShape(12.dp),
+ leadingIcon = {
+ Icon(
+ imageVector = androidx.compose.material.icons.Icons.Default.Info,
+ contentDescription = "Search"
+ )
+ }
+ )
+
+ LazyColumn(
+ modifier = Modifier
+ .fillMaxWidth()
+ .weight(1f)
+ .padding(horizontal = 16.dp, vertical = 8.dp)
+ ) {
+ items(filteredLanguages) { lang ->
+ val langCode = lang.second
+ val rawName = lang.first
+ val displayName = remember(rawName) {
+ formatLanguageName(rawName)
+ }
+ val isDefault = langCode == activeLanguageCode
+ val isDownloaded = langCode in downloadedLanguages || langCode == "en"
+ val isDownloading = langCode in downloadingLanguages
+
+ Surface(
+ shape = RoundedCornerShape(12.dp),
+ color = if (isDefault) MaterialTheme.colorScheme.primaryContainer.copy(alpha = 0.1f) else Color.Transparent,
+ border = if (isDefault) BorderStroke(1.dp, MaterialTheme.colorScheme.primary.copy(alpha = 0.4f)) else null,
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 4.dp)
+ .clickable {
+ if (isDownloaded) {
+ onLanguageSelected(langCode)
+ } else if (!isDownloading) {
+ onDownloadLanguage(langCode)
+ onLanguageSelected(langCode)
+ }
+ }
+ ) {
+ Row(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(12.dp),
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ RadioButton(
+ selected = isDefault,
+ onClick = {
+ if (isDownloaded) {
+ onLanguageSelected(langCode)
+ } else if (!isDownloading) {
+ onDownloadLanguage(langCode)
+ onLanguageSelected(langCode)
+ }
+ }
+ )
+
+ Spacer(modifier = Modifier.width(12.dp))
+
+ Column(modifier = Modifier.weight(1f)) {
+ Text(
+ text = displayName,
+ style = MaterialTheme.typography.bodyLarge,
+ fontWeight = if (isDefault) FontWeight.Bold else FontWeight.Normal
+ )
+ Text(
+ text = "Code: $langCode",
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.7f)
+ )
+ }
+
+ Spacer(modifier = Modifier.width(8.dp))
+
+ when {
+ isDownloading -> {
+ CircularProgressIndicator(
+ modifier = Modifier.size(20.dp),
+ strokeWidth = 2.dp
+ )
+ }
+ isDownloaded -> {
+ Row(verticalAlignment = Alignment.CenterVertically) {
+ StatusBadge(text = "Installed", isInstalled = true)
+ Spacer(modifier = Modifier.width(8.dp))
+ if (langCode != "en") {
+ IconButton(onClick = { onDeleteLanguage(langCode) }) {
+ Icon(
+ imageVector = androidx.compose.material.icons.Icons.Default.Delete,
+ contentDescription = "Delete model",
+ tint = MaterialTheme.colorScheme.error
+ )
+ }
+ }
+ }
+ }
+ else -> {
+ IconButton(
+ onClick = {
+ onDownloadLanguage(langCode)
+ onLanguageSelected(langCode)
+ }
+ ) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_download),
+ contentDescription = "Download model",
+ tint = MaterialTheme.colorScheme.primary
+ )
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+}
+
+fun formatLanguageName(name: String): String {
+ return name.lowercase().split("_").joinToString(" ") { it.replaceFirstChar { char -> char.uppercase() } }
+}
+
+private fun openAssistantSettings(context: Context) {
+ try {
+ val intent = Intent(Settings.ACTION_VOICE_INPUT_SETTINGS)
+ intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
+ context.startActivity(intent)
+ } catch (e: Exception) {
+ try {
+ val intent = Intent(Settings.ACTION_MANAGE_DEFAULT_APPS_SETTINGS)
+ intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
+ context.startActivity(intent)
+ } catch (ex: Exception) {
+ try {
+ val intent = Intent(Settings.ACTION_SETTINGS)
+ intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
+ context.startActivity(intent)
+ } catch (exc: Exception) {
+ // Ignore
+ }
+ }
+ }
+}
+
+
diff --git a/app/src/main/java/com/app/assistant/ui/screen/UserInputField.kt b/app/src/main/java/com/app/assistant/ui/screen/UserInputField.kt
new file mode 100644
index 0000000..80eb64e
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/ui/screen/UserInputField.kt
@@ -0,0 +1,463 @@
+package com.app.assistant.ui.screen
+
+import androidx.compose.animation.core.Animatable
+import androidx.compose.animation.core.LinearEasing
+import androidx.compose.animation.core.tween
+import androidx.compose.animation.AnimatedVisibility
+import androidx.compose.animation.fadeIn
+import androidx.compose.animation.fadeOut
+import androidx.compose.animation.expandHorizontally
+import androidx.compose.animation.shrinkHorizontally
+import androidx.compose.foundation.Canvas
+import androidx.compose.foundation.Image
+import androidx.compose.foundation.background
+import androidx.compose.ui.draw.clip
+import androidx.compose.foundation.layout.Arrangement
+import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.Column
+import androidx.compose.foundation.layout.Row
+import androidx.compose.foundation.layout.Spacer
+import androidx.compose.foundation.layout.fillMaxSize
+import androidx.compose.foundation.layout.fillMaxWidth
+import androidx.compose.foundation.layout.height
+import androidx.compose.foundation.layout.imePadding
+import androidx.compose.foundation.layout.padding
+import androidx.compose.foundation.layout.size
+import androidx.compose.foundation.lazy.LazyRow
+import androidx.compose.foundation.lazy.items
+import androidx.compose.foundation.shape.CircleShape
+import androidx.compose.foundation.shape.RoundedCornerShape
+import androidx.compose.material.icons.Icons
+import androidx.compose.material.icons.automirrored.filled.Send
+import androidx.compose.material.icons.filled.Close
+import androidx.compose.material3.CircularProgressIndicator
+import androidx.compose.material3.DropdownMenu
+import androidx.compose.material3.DropdownMenuItem
+import androidx.compose.material3.Icon
+import androidx.compose.material3.IconButton
+import androidx.compose.material3.MaterialTheme
+import androidx.compose.material3.Text
+import androidx.compose.material3.TextField
+import androidx.compose.material3.TextFieldDefaults
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.LaunchedEffect
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.mutableStateOf
+import androidx.compose.runtime.remember
+import androidx.compose.runtime.setValue
+import androidx.compose.ui.Alignment
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.graphics.asImageBitmap
+import androidx.compose.ui.graphics.luminance
+import androidx.compose.ui.layout.ContentScale
+import androidx.compose.ui.layout.onGloballyPositioned
+import androidx.compose.ui.platform.LocalContext
+import androidx.compose.ui.platform.SoftwareKeyboardController
+import androidx.compose.ui.res.painterResource
+import androidx.compose.ui.res.stringResource
+import androidx.compose.ui.text.style.TextAlign
+import androidx.compose.ui.text.style.TextOverflow
+import androidx.compose.ui.tooling.preview.Preview
+import androidx.compose.ui.unit.IntSize
+import androidx.compose.ui.unit.dp
+import androidx.compose.ui.unit.sp
+import com.app.assistant.R
+import com.app.assistant.model.Attachment
+import com.app.assistant.ui.theme.AssistantTheme
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.currentCoroutineContext
+import kotlinx.coroutines.isActive
+import kotlinx.coroutines.launch
+import kotlinx.coroutines.withContext
+import kotlin.math.hypot
+
+@Composable
+@Suppress("ktlint:standard:function-naming")
+fun UserInputField(
+ focusManager: androidx.compose.ui.focus.FocusManager,
+ keyboardController: SoftwareKeyboardController,
+ isSpeaking: Boolean,
+ isListening: Boolean,
+ question: String,
+ onQuestionChange: (String) -> Unit,
+ onStopSpeaking: () -> Unit,
+ onStartListening: () -> Unit,
+ onProcessQuestion: (androidx.compose.ui.focus.FocusManager, SoftwareKeyboardController, Boolean) -> Unit,
+ modifier: Modifier = Modifier,
+ selectedAttachments: List = emptyList(),
+ onRemoveAttachment: (Attachment) -> Unit = {},
+ onAttachClick: (String) -> Unit = {},
+ isImageSupported: Boolean = false,
+ isAudioSupported: Boolean = false,
+ isVideoSupported: Boolean = false,
+ isDocumentSupported: Boolean = false,
+ isHandsFree: Boolean = false,
+ onToggleHandsFree: () -> Unit = {},
+ hasMessages: Boolean = false,
+) {
+ val containerColor = TextFieldDefaults.colors().unfocusedContainerColor
+ val rippleColor = adjustContrast(containerColor)
+
+ // State for current radius + alpha
+ val rippleRadius = remember { Animatable(0f) }
+ val rippleAlpha = remember { Animatable(0f) }
+
+ // Size of TextField (for max ripple radius)
+ var textFieldSize by remember { mutableStateOf(IntSize.Zero) }
+
+ LaunchedEffect(isListening, textFieldSize) {
+ if (isListening && textFieldSize != IntSize.Zero) {
+ val maxRadius =
+ hypot(
+ textFieldSize.width / 2f,
+ textFieldSize.height / 2f,
+ )
+ while (currentCoroutineContext().isActive) { // keep running until LaunchedEffect cancels
+ rippleRadius.snapTo(0f)
+ rippleAlpha.snapTo(0.5f)
+
+ // animate simultaneously
+ launch {
+ rippleRadius.animateTo(
+ targetValue = maxRadius,
+ animationSpec = tween(1000, easing = LinearEasing),
+ )
+ }
+ rippleAlpha.animateTo(
+ targetValue = 0f,
+ animationSpec = tween(1000, easing = LinearEasing),
+ )
+ }
+ } else {
+ rippleRadius.snapTo(0f)
+ rippleAlpha.snapTo(0f)
+ }
+ }
+
+ Column(
+ modifier = modifier
+ .fillMaxWidth()
+ .imePadding()
+ .padding(8.dp)
+ ) {
+ if (selectedAttachments.isNotEmpty()) {
+ AttachmentPreviewList(
+ attachments = selectedAttachments,
+ onRemove = onRemoveAttachment,
+ modifier = Modifier.padding(bottom = 8.dp)
+ )
+ }
+
+ Box(
+ modifier = Modifier
+ .fillMaxWidth()
+ .clip(RoundedCornerShape(20)) // clip ripple inside
+ .onGloballyPositioned { coords -> textFieldSize = coords.size },
+ ) {
+ TextField(
+ modifier = Modifier.fillMaxWidth(),
+ shape = RoundedCornerShape(20),
+ value = question,
+ placeholder = { Text(stringResource(id = R.string.type_here_placeholder)) },
+ maxLines = 4,
+ onValueChange = onQuestionChange,
+ colors = TextFieldDefaults.colors(
+ focusedIndicatorColor = Color.Transparent,
+ unfocusedIndicatorColor = Color.Transparent,
+ disabledIndicatorColor = Color.Transparent,
+ ),
+ leadingIcon = if ((isImageSupported || isAudioSupported || isVideoSupported || isDocumentSupported) && !isSpeaking) {
+ {
+ var menuExpanded by remember { mutableStateOf(false) }
+ Box {
+ IconButton(onClick = { menuExpanded = true }) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_attachment),
+ contentDescription = "Attach file",
+ tint = MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ }
+ DropdownMenu(
+ expanded = menuExpanded,
+ onDismissRequest = { menuExpanded = false }
+ ) {
+ if (isImageSupported) {
+ DropdownMenuItem(
+ text = { Text("Image") },
+ onClick = {
+ menuExpanded = false
+ onAttachClick("image/*")
+ }
+ )
+ DropdownMenuItem(
+ text = { Text("Take Photo (Camera)") },
+ onClick = {
+ menuExpanded = false
+ onAttachClick("camera")
+ }
+ )
+ }
+ if (isAudioSupported) {
+ DropdownMenuItem(
+ text = { Text("Audio") },
+ onClick = {
+ menuExpanded = false
+ onAttachClick("audio/*")
+ }
+ )
+ }
+ if (isVideoSupported) {
+ DropdownMenuItem(
+ text = { Text("Video") },
+ onClick = {
+ menuExpanded = false
+ onAttachClick("video/*")
+ }
+ )
+ }
+ if (isDocumentSupported) {
+ DropdownMenuItem(
+ text = { Text("Document (PDF)") },
+ onClick = {
+ menuExpanded = false
+ onAttachClick("application/pdf")
+ }
+ )
+ }
+ // Always support text files as they are decrypted and appended inline as text
+ DropdownMenuItem(
+ text = { Text("Text Document (.txt/.csv/.json)") },
+ onClick = {
+ menuExpanded = false
+ onAttachClick("text/*")
+ }
+ )
+ }
+ }
+ }
+ } else null,
+ trailingIcon = {
+ val hasInput = question.isNotEmpty() || selectedAttachments.isNotEmpty()
+ val showSendButton = hasInput || isListening || hasMessages
+ Row(
+ horizontalArrangement = Arrangement.End,
+ verticalAlignment = Alignment.CenterVertically
+ ) {
+ if (isSpeaking) {
+ IconButton(onClick = {
+ if (isSpeaking) {
+ onStopSpeaking()
+ }
+ }) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_stop),
+ contentDescription = stringResource(id = R.string.stop_desc),
+ )
+ }
+ } else {
+ IconButton(onClick = {
+ onStartListening()
+ }) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_mic),
+ contentDescription = stringResource(id = R.string.mic_desc),
+ )
+ }
+ }
+ AnimatedVisibility(
+ visible = !showSendButton,
+ enter = fadeIn() + expandHorizontally(),
+ exit = fadeOut() + shrinkHorizontally()
+ ) {
+ IconButton(onClick = onToggleHandsFree) {
+ Icon(
+ painter = painterResource(id = R.drawable.ic_live_modes),
+ contentDescription = "Toggle Hands-free Mode",
+ tint = if (isHandsFree) MaterialTheme.colorScheme.primary else MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ }
+ }
+ AnimatedVisibility(
+ visible = showSendButton,
+ enter = fadeIn() + expandHorizontally(),
+ exit = fadeOut() + shrinkHorizontally()
+ ) {
+ IconButton(onClick = {
+ if (question.isNotEmpty() || selectedAttachments.isNotEmpty()) {
+ onProcessQuestion(focusManager, keyboardController, false)
+ }
+ }) {
+ Icon(
+ imageVector = Icons.AutoMirrored.Filled.Send,
+ contentDescription = stringResource(id = R.string.send_desc),
+ )
+ }
+ }
+ }
+ },
+ )
+
+ Canvas(
+ modifier = Modifier
+ .matchParentSize()
+ .clip(RoundedCornerShape(20)),
+ ) {
+ if (rippleAlpha.value > 0f) {
+ drawCircle(
+ color = rippleColor.copy(alpha = rippleAlpha.value),
+ radius = rippleRadius.value,
+ center = center,
+ )
+ }
+ }
+ }
+ }
+}
+
+@Composable
+fun AttachmentPreviewList(
+ attachments: List,
+ onRemove: (Attachment) -> Unit,
+ modifier: Modifier = Modifier
+) {
+ LazyRow(
+ modifier = modifier.fillMaxWidth(),
+ horizontalArrangement = Arrangement.spacedBy(8.dp)
+ ) {
+ items(attachments) { attachment ->
+ AttachmentPreviewCard(attachment = attachment, onRemove = { onRemove(attachment) })
+ }
+ }
+}
+
+@Composable
+fun AttachmentPreviewCard(
+ attachment: Attachment,
+ onRemove: () -> Unit
+) {
+ Box(
+ modifier = Modifier
+ .size(72.dp)
+ .clip(RoundedCornerShape(8.dp))
+ .background(MaterialTheme.colorScheme.surfaceVariant)
+ ) {
+ if (attachment.mimeType.startsWith("image/")) {
+ var bitmap by remember { mutableStateOf(null) }
+ LaunchedEffect(attachment) {
+ withContext(Dispatchers.IO) {
+ try {
+ val fileBytes = java.io.File(attachment.filePath).readBytes()
+ val decrypted = com.app.assistant.db.EncryptionUtil.decryptFile(fileBytes, attachment.iv)
+ bitmap = android.graphics.BitmapFactory.decodeByteArray(decrypted, 0, decrypted.size)
+ } catch (e: Exception) {
+ e.printStackTrace()
+ }
+ }
+ }
+ if (bitmap != null) {
+ Image(
+ bitmap = bitmap!!.asImageBitmap(),
+ contentDescription = null,
+ modifier = Modifier.fillMaxSize(),
+ contentScale = ContentScale.Crop
+ )
+ } else {
+ Box(modifier = Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
+ CircularProgressIndicator(modifier = Modifier.size(16.dp), strokeWidth = 2.dp)
+ }
+ }
+ } else {
+ Column(
+ modifier = Modifier
+ .fillMaxSize()
+ .padding(4.dp),
+ horizontalAlignment = Alignment.CenterHorizontally,
+ verticalArrangement = Arrangement.Center
+ ) {
+ val iconRes = when {
+ attachment.mimeType.startsWith("audio/") -> R.drawable.ic_mic
+ attachment.mimeType.startsWith("video/") -> R.drawable.ic_stop
+ else -> R.drawable.ic_translate
+ }
+ Icon(
+ painter = painterResource(id = iconRes),
+ contentDescription = null,
+ modifier = Modifier.size(24.dp),
+ tint = MaterialTheme.colorScheme.primary
+ )
+ Spacer(modifier = Modifier.height(2.dp))
+ Text(
+ text = attachment.fileName,
+ fontSize = 8.sp,
+ maxLines = 1,
+ overflow = TextOverflow.Ellipsis,
+ textAlign = TextAlign.Center,
+ color = MaterialTheme.colorScheme.onSurfaceVariant
+ )
+ }
+ }
+
+ IconButton(
+ onClick = onRemove,
+ modifier = Modifier
+ .align(Alignment.TopEnd)
+ .size(18.dp)
+ .background(Color.Black.copy(alpha = 0.6f), shape = CircleShape)
+ .padding(2.dp)
+ ) {
+ Icon(
+ imageVector = Icons.Default.Close,
+ contentDescription = "Remove",
+ tint = Color.White,
+ modifier = Modifier.size(12.dp)
+ )
+ }
+ }
+}
+
+fun adjustContrast(
+ color: Color,
+ factor: Float = 0.15f,
+): Color {
+ val isDark = color.luminance() < 0.5
+ return if (isDark) {
+ Color(
+ red = (color.red + factor).coerceAtMost(1f),
+ green = (color.green + factor).coerceAtMost(1f),
+ blue = (color.blue + factor).coerceAtMost(1f),
+ alpha = color.alpha,
+ )
+ } else {
+ Color(
+ red = (color.red - factor).coerceAtLeast(0f),
+ green = (color.green - factor).coerceAtLeast(0f),
+ blue = (color.blue - factor).coerceAtLeast(0f),
+ alpha = color.alpha,
+ )
+ }
+}
+
+@Preview(showBackground = true)
+@Composable
+fun UserInputFieldIdlePreview() {
+ val focusManager = androidx.compose.ui.platform.LocalFocusManager.current
+ val keyboardController = androidx.compose.ui.platform.LocalSoftwareKeyboardController.current
+ AssistantTheme {
+ if (keyboardController != null) {
+ UserInputField(
+ focusManager = focusManager,
+ keyboardController = keyboardController,
+ isSpeaking = false,
+ isListening = false,
+ question = "",
+ onQuestionChange = {},
+ onStopSpeaking = {},
+ onStartListening = {},
+ onProcessQuestion = { _, _, _ -> },
+ isImageSupported = true,
+ hasMessages = true
+ )
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/ui/theme/Color.kt b/app/src/main/java/com/app/assistant/ui/theme/Color.kt
index 05ffa41..581271f 100644
--- a/app/src/main/java/com/app/assistant/ui/theme/Color.kt
+++ b/app/src/main/java/com/app/assistant/ui/theme/Color.kt
@@ -8,4 +8,4 @@ val Pink80 = Color(0xFFEFB8C8)
val Purple40 = Color(0xFF6650a4)
val PurpleGrey40 = Color(0xFF625b71)
-val Pink40 = Color(0xFF7D5260)
\ No newline at end of file
+val Pink40 = Color(0xFF7D5260)
diff --git a/app/src/main/java/com/app/assistant/ui/theme/Theme.kt b/app/src/main/java/com/app/assistant/ui/theme/Theme.kt
index fa44c69..55a67a5 100644
--- a/app/src/main/java/com/app/assistant/ui/theme/Theme.kt
+++ b/app/src/main/java/com/app/assistant/ui/theme/Theme.kt
@@ -11,17 +11,18 @@ import androidx.compose.material3.lightColorScheme
import androidx.compose.runtime.Composable
import androidx.compose.ui.platform.LocalContext
-private val DarkColorScheme = darkColorScheme(
- primary = Purple80,
- secondary = PurpleGrey80,
- tertiary = Pink80
-)
-
-private val LightColorScheme = lightColorScheme(
- primary = Purple40,
- secondary = PurpleGrey40,
- tertiary = Pink40
+private val DarkColorScheme =
+ darkColorScheme(
+ primary = Purple80,
+ secondary = PurpleGrey80,
+ tertiary = Pink80,
+ )
+private val LightColorScheme =
+ lightColorScheme(
+ primary = Purple40,
+ secondary = PurpleGrey40,
+ tertiary = Pink40,
/* Other default colors to override
background = Color(0xFFFFFBFE),
surface = Color(0xFFFFFBFE),
@@ -30,28 +31,35 @@ private val LightColorScheme = lightColorScheme(
onTertiary = Color.White,
onBackground = Color(0xFF1C1B1F),
onSurface = Color(0xFF1C1B1F),
- */
-)
+ */
+ )
@Composable
fun AssistantTheme(
darkTheme: Boolean = isSystemInDarkTheme(),
// Dynamic color is available on Android 12+
dynamicColor: Boolean = true,
- content: @Composable () -> Unit
+ content: @Composable () -> Unit,
) {
- val colorScheme = when {
- dynamicColor && Build.VERSION.SDK_INT >= Build.VERSION_CODES.S -> {
- val context = LocalContext.current
- if (darkTheme) dynamicDarkColorScheme(context) else dynamicLightColorScheme(context)
- }
- darkTheme -> DarkColorScheme
- else -> LightColorScheme
- }
+ val colorScheme =
+ when {
+ dynamicColor && Build.VERSION.SDK_INT >= Build.VERSION_CODES.S -> {
+ val context = LocalContext.current
+ if (darkTheme) dynamicDarkColorScheme(context) else dynamicLightColorScheme(context)
+ }
+
+ darkTheme -> {
+ DarkColorScheme
+ }
+
+ else -> {
+ LightColorScheme
+ }
+ }
MaterialTheme(
- colorScheme = colorScheme,
- typography = Typography,
- content = content
+ colorScheme = colorScheme,
+ typography = Typography,
+ content = content,
)
-}
\ No newline at end of file
+}
diff --git a/app/src/main/java/com/app/assistant/ui/theme/Type.kt b/app/src/main/java/com/app/assistant/ui/theme/Type.kt
index 22515d0..1ab9d3f 100644
--- a/app/src/main/java/com/app/assistant/ui/theme/Type.kt
+++ b/app/src/main/java/com/app/assistant/ui/theme/Type.kt
@@ -7,14 +7,16 @@ import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.sp
// Set of Material typography styles to start with
-val Typography = Typography(
- bodyLarge = TextStyle(
- fontFamily = FontFamily.Default,
- fontWeight = FontWeight.Normal,
- fontSize = 16.sp,
- lineHeight = 24.sp,
- letterSpacing = 0.5.sp
- )
+val Typography =
+ Typography(
+ bodyLarge =
+ TextStyle(
+ fontFamily = FontFamily.Default,
+ fontWeight = FontWeight.Normal,
+ fontSize = 16.sp,
+ lineHeight = 24.sp,
+ letterSpacing = 0.5.sp,
+ ),
/* Other default text styles to override
titleLarge = TextStyle(
fontFamily = FontFamily.Default,
@@ -30,5 +32,5 @@ val Typography = Typography(
lineHeight = 16.sp,
letterSpacing = 0.5.sp
)
- */
-)
\ No newline at end of file
+ */
+ )
diff --git a/app/src/main/java/com/app/assistant/usecase/CallContactUseCase.kt b/app/src/main/java/com/app/assistant/usecase/CallContactUseCase.kt
new file mode 100644
index 0000000..a01ec3b
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/CallContactUseCase.kt
@@ -0,0 +1,44 @@
+package com.app.assistant.usecase
+
+import com.app.assistant.model.DeviceAction
+import com.app.assistant.repository.ContactsRepository
+import java.net.URI
+
+class CallContactUseCase(
+ private val permissionChecker: PermissionChecker,
+ private val contactsRepository: ContactsRepository
+) {
+ suspend fun execute(
+ prompt: String,
+ onPermissionRequest: suspend (Array) -> Unit,
+ onIntentTriggered: suspend (DeviceAction) -> Unit,
+ onSuccess: suspend (name: String, dialUri: URI) -> Unit,
+ onFailure: suspend (errorMsg: String) -> Unit
+ ) {
+ val requiredPermissions = arrayOf(
+ "android.permission.READ_CONTACTS",
+ "android.permission.CALL_PHONE"
+ ).filter {
+ !permissionChecker.hasPermission(it)
+ }
+
+ if (requiredPermissions.isNotEmpty()) {
+ onPermissionRequest(requiredPermissions.toTypedArray())
+ return
+ }
+
+ val bestMatch = contactsRepository.searchContact(prompt)
+ if (bestMatch != null) {
+ try {
+ val callAction = DeviceAction.MakeCall(bestMatch.phoneNumber)
+ onIntentTriggered(callAction)
+ onSuccess(bestMatch.name, URI("tel:${bestMatch.phoneNumber.replace(" ", "")}"))
+ } catch (e: Exception) {
+ System.err.println("Error making call: ${e.message}")
+ onFailure("Sorry, failed to make call. Please try again.")
+ }
+ } else {
+ onFailure("I cannot find such contact, please try again.")
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/GetAiResponseUseCase.kt b/app/src/main/java/com/app/assistant/usecase/GetAiResponseUseCase.kt
new file mode 100644
index 0000000..8ac2d75
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/GetAiResponseUseCase.kt
@@ -0,0 +1,118 @@
+package com.app.assistant.usecase
+
+import com.app.assistant.llm.FlexibleLlmAdapter
+import com.app.assistant.llm.LlmConfig
+import com.app.assistant.llm.LlmMessage
+import com.app.assistant.llm.LlmProvider
+import com.app.assistant.repository.SettingsRepository
+import kotlinx.coroutines.flow.Flow
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.JsonObject
+import kotlinx.serialization.json.JsonPrimitive
+import okhttp3.OkHttpClient
+
+class GetAiResponseUseCase(
+ private val settingsRepository: SettingsRepository,
+ private val client: OkHttpClient
+) {
+ private val json = Json { ignoreUnknownKeys = true }
+
+ suspend fun execute(messages: List): String? {
+ val providerName = settingsRepository.getLlmProvider()
+ val provider = try {
+ LlmProvider.valueOf(providerName)
+ } catch (e: Exception) {
+ LlmProvider.GROQ
+ }
+
+ var apiKey = settingsRepository.getChatApiKey()
+ if (apiKey.isNullOrBlank() && provider == LlmProvider.GROQ) {
+ apiKey = com.app.assistant.BuildConfig.GROQ_API_KEY
+ }
+
+ val model = settingsRepository.getLlmModel()
+
+ val config = if (provider == LlmProvider.CUSTOM) {
+ LlmConfig(
+ url = settingsRepository.getLlmCustomUrl(),
+ headers = parseHeadersJson(settingsRepository.getLlmCustomHeaders()),
+ responsePath = settingsRepository.getLlmCustomResponsePath(),
+ requestTemplate = settingsRepository.getLlmCustomRequestTemplate(),
+ messageFormat = settingsRepository.getLlmCustomMessageFormat(),
+ systemRole = settingsRepository.getLlmCustomSystemRole().takeIf { it.isNotBlank() },
+ userRole = settingsRepository.getLlmCustomUserRole(),
+ assistantRole = settingsRepository.getLlmCustomAssistantRole()
+ )
+ } else {
+ provider.config
+ }
+
+ val adapter = FlexibleLlmAdapter(
+ client = client,
+ apiKey = apiKey ?: "",
+ model = model,
+ config = config,
+ provider = provider
+ )
+
+ val systemContext = messages.find { it.role == "system" }?.content ?: ""
+ return adapter.generateResponse(systemContext, messages)
+ }
+
+ fun executeStream(messages: List): Flow {
+ val providerName = settingsRepository.getLlmProvider()
+ val provider = try {
+ LlmProvider.valueOf(providerName)
+ } catch (e: Exception) {
+ LlmProvider.GROQ
+ }
+
+ var apiKey = settingsRepository.getChatApiKey()
+ if (apiKey.isNullOrBlank() && provider == LlmProvider.GROQ) {
+ apiKey = com.app.assistant.BuildConfig.GROQ_API_KEY
+ }
+
+ val model = settingsRepository.getLlmModel()
+
+ val config = if (provider == LlmProvider.CUSTOM) {
+ LlmConfig(
+ url = settingsRepository.getLlmCustomUrl(),
+ headers = parseHeadersJson(settingsRepository.getLlmCustomHeaders()),
+ responsePath = settingsRepository.getLlmCustomResponsePath(),
+ requestTemplate = settingsRepository.getLlmCustomRequestTemplate(),
+ messageFormat = settingsRepository.getLlmCustomMessageFormat(),
+ systemRole = settingsRepository.getLlmCustomSystemRole().takeIf { it.isNotBlank() },
+ userRole = settingsRepository.getLlmCustomUserRole(),
+ assistantRole = settingsRepository.getLlmCustomAssistantRole()
+ )
+ } else {
+ provider.config
+ }
+
+ val adapter = FlexibleLlmAdapter(
+ client = client,
+ apiKey = apiKey ?: "",
+ model = model,
+ config = config,
+ provider = provider
+ )
+
+ val systemContext = messages.find { it.role == "system" }?.content ?: ""
+ return adapter.generateResponseStream(systemContext, messages)
+ }
+
+ private fun parseHeadersJson(jsonStr: String): Map {
+ return try {
+ val element = json.parseToJsonElement(jsonStr)
+ if (element is JsonObject) {
+ element.mapValues { (_, value) ->
+ if (value is JsonPrimitive) value.content else value.toString()
+ }
+ } else {
+ emptyMap()
+ }
+ } catch (e: Exception) {
+ emptyMap()
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/GetWeatherUseCase.kt b/app/src/main/java/com/app/assistant/usecase/GetWeatherUseCase.kt
new file mode 100644
index 0000000..7ebdfa6
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/GetWeatherUseCase.kt
@@ -0,0 +1,103 @@
+package com.app.assistant.usecase
+
+import com.app.assistant.llm.LlmMessage
+import com.app.assistant.repository.WeatherRepository
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.JsonObject
+import kotlinx.serialization.json.jsonObject
+import java.text.SimpleDateFormat
+import java.util.Calendar
+import java.util.Locale
+
+class GetWeatherUseCase(
+ private val permissionChecker: PermissionChecker,
+ private val weatherRepository: WeatherRepository,
+ private val getAiResponseUseCase: GetAiResponseUseCase
+) {
+ suspend fun execute(
+ prompt: String,
+ onPermissionRequest: suspend (Array) -> Unit,
+ onLocationRequest: suspend () -> Unit,
+ onSuccess: suspend (response: String, city: String) -> Unit,
+ onFailure: suspend (errorMsg: String) -> Unit
+ ) {
+ try {
+ val location = weatherRepository.extractPlaceName(prompt)
+ if (location.isNotEmpty()) {
+ val coordinates = weatherRepository.getCoordinatesFromCity(location)
+ if (coordinates != null) {
+ val (lat, long) = coordinates
+ val weatherData = weatherRepository.getWeatherData(lat, long)
+ if (weatherData != null) {
+ val weatherJson = Json.parseToJsonElement(weatherData).jsonObject
+ val filteredMap = weatherJson.filterKeys { it != "latitude" && it != "longitude" }
+ val filteredJson = JsonObject(filteredMap)
+
+ val aiResponse = queryWeatherAI(prompt, filteredJson.toString())
+ if (!aiResponse.isNullOrEmpty()) {
+ onSuccess(aiResponse, location)
+ } else {
+ onFailure("Seems weather report is not available, please try again.")
+ }
+ } else {
+ onFailure("Seems weather report is not available, please try again.")
+ }
+ } else {
+ onFailure("I can not find such location, please try again.")
+ }
+ } else {
+ val isPermissionGranted = permissionChecker.hasPermission("android.permission.ACCESS_COARSE_LOCATION")
+
+ if (!isPermissionGranted) {
+ onPermissionRequest(arrayOf("android.permission.ACCESS_COARSE_LOCATION"))
+ } else {
+ onLocationRequest()
+ }
+ }
+ } catch (e: Exception) {
+ System.err.println("Error fetching weather: ${e.message}")
+ onFailure("Something went wrong, please try again.")
+ }
+ }
+
+ suspend fun processLocationWeather(
+ lat: Double,
+ long: Double,
+ prompt: String,
+ onSuccess: suspend (response: String, city: String) -> Unit,
+ onFailure: suspend (errorMsg: String) -> Unit
+ ) {
+ val city = weatherRepository.getCityNameFromLocation(lat, long) ?: "your location"
+ val weatherData = weatherRepository.getWeatherData(lat, long)
+ if (weatherData != null) {
+ val weatherJson = Json.parseToJsonElement(weatherData).jsonObject
+ val filteredMap = weatherJson.filterKeys { it != "latitude" && it != "longitude" }
+ val filteredJson = JsonObject(filteredMap)
+
+ val aiResponse = queryWeatherAI(prompt, filteredJson.toString())
+ if (!aiResponse.isNullOrEmpty()) {
+ onSuccess(aiResponse, city)
+ } else {
+ onFailure("Seems weather report is not available, please try again.")
+ }
+ } else {
+ onFailure("Seems weather report is not available, please try again.")
+ }
+ }
+
+ private suspend fun queryWeatherAI(question: String, weatherData: String): String? {
+ val date = SimpleDateFormat("yyyy-MM-dd", Locale.getDefault(Locale.Category.FORMAT)).format(Calendar.getInstance().time)
+ val time = SimpleDateFormat("HH:mm:ss", Locale.getDefault(Locale.Category.FORMAT)).format(Calendar.getInstance().time)
+ val systemContext =
+ "You are a smart weather assistant, up-to-date with the current date and time, " +
+ "which is $date at $time. Please respond with an answer to the user's question based on " +
+ "the latest weather data provided. No need to mention data or time; just answer naturally in 2 to 3 lines."
+
+ val messages = listOf(
+ LlmMessage(role = "system", content = systemContext),
+ LlmMessage(role = "user", content = question + System.lineSeparator() + weatherData)
+ )
+
+ return getAiResponseUseCase.execute(messages)
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/NavigateUseCase.kt b/app/src/main/java/com/app/assistant/usecase/NavigateUseCase.kt
new file mode 100644
index 0000000..d80eaa3
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/NavigateUseCase.kt
@@ -0,0 +1,44 @@
+package com.app.assistant.usecase
+
+import com.app.assistant.model.DeviceAction
+import java.net.URI
+
+class NavigateUseCase(private val resourceProvider: ResourceProvider) {
+ suspend fun execute(
+ prompt: String,
+ onIntentTriggered: suspend (DeviceAction) -> Unit,
+ onSuccess: suspend (location: String, navigationUri: URI) -> Unit,
+ onFailure: suspend (errorMsg: String) -> Unit
+ ) {
+ try {
+ val sanitizedPrompt = prompt.replace("\\p{Punct}+".toRegex(), "")
+ val location = extractLocation(sanitizedPrompt)
+ if (location != null) {
+ val encodedURI = java.net.URLEncoder.encode(
+ location,
+ java.nio.charset.StandardCharsets.UTF_8.toString()
+ )
+ onIntentTriggered(DeviceAction.NavigateTo(location))
+ onSuccess(location, URI("google.navigation:q=$encodedURI"))
+ } else {
+ onFailure(resourceProvider.getString("location_not_found"))
+ }
+ } catch (e: Exception) {
+ System.err.println("Error navigating: ${e.message}")
+ onFailure(resourceProvider.getString("generic_error"))
+ }
+ }
+
+ private fun extractLocation(command: String): String? {
+ val regex = Regex(
+ "(?<=to |find |show me |give me directions to |navigate me to |" +
+ "how do I get to |I'm on my way to |start a navigation to |" +
+ "where is |help me find |traffic like on the way to |way to )" +
+ "([A-Za-z0-9\\s&]+)",
+ RegexOption.IGNORE_CASE
+ )
+
+ val matchResult = regex.find(command)
+ return matchResult?.value?.trim()
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/PermissionChecker.kt b/app/src/main/java/com/app/assistant/usecase/PermissionChecker.kt
new file mode 100644
index 0000000..c9c1876
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/PermissionChecker.kt
@@ -0,0 +1,5 @@
+package com.app.assistant.usecase
+
+interface PermissionChecker {
+ fun hasPermission(permission: String): Boolean
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/PlaySongUseCase.kt b/app/src/main/java/com/app/assistant/usecase/PlaySongUseCase.kt
new file mode 100644
index 0000000..f5864c2
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/PlaySongUseCase.kt
@@ -0,0 +1,164 @@
+package com.app.assistant.usecase
+
+import com.app.assistant.BuildConfig
+import com.app.assistant.model.DeviceAction
+import com.app.assistant.repository.SettingsRepository
+import kotlinx.coroutines.CompletableDeferred
+import kotlinx.serialization.Serializable
+import kotlinx.serialization.json.Json
+import okhttp3.Call
+import okhttp3.Callback
+import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
+import okhttp3.OkHttpClient
+import okhttp3.Request
+import okhttp3.Response
+import java.io.IOException
+import java.net.URI
+
+@Serializable
+data class YouTubeItemId(
+ val kind: String? = null,
+ val videoId: String? = null
+)
+
+@Serializable
+data class YouTubeThumbnailDetail(
+ val url: String? = null
+)
+
+@Serializable
+data class YouTubeThumbnails(
+ val high: YouTubeThumbnailDetail? = null
+)
+
+@Serializable
+data class YouTubeSnippet(
+ val thumbnails: YouTubeThumbnails? = null
+)
+
+@Serializable
+data class YouTubeSearchItem(
+ val id: YouTubeItemId? = null,
+ val snippet: YouTubeSnippet? = null
+)
+
+@Serializable
+data class YouTubeSearchResponse(
+ val items: List? = null
+)
+
+class PlaySongUseCase(
+ private val resourceProvider: ResourceProvider,
+ private val settingsRepository: SettingsRepository,
+ private val client: OkHttpClient
+) {
+ private val json = Json { ignoreUnknownKeys = true }
+
+ suspend fun execute(
+ prompt: String,
+ onIntentTriggered: suspend (DeviceAction) -> Unit,
+ onSuccess: suspend (songName: String, videoId: String, thumbnailUrl: String, videoUri: URI) -> Unit,
+ onMissingApiKey: suspend (searchQuery: String) -> Unit,
+ onFailure: suspend (errorMsg: String) -> Unit
+ ) {
+ try {
+ val sanitizedPrompt = prompt.replace("\\p{Punct}+".toRegex(), "")
+ if (!sanitizedPrompt.lowercase().contains("play ")) {
+ onFailure(resourceProvider.getString("song_not_found"))
+ return
+ }
+ val searchQuery = sanitizedPrompt.lowercase().substringAfter("play").trim()
+
+ val (videoId, thumbnailUrl) = youtubeApiCall(searchQuery)
+
+ if (videoId.isEmpty()) {
+ onFailure(resourceProvider.getString("song_not_found"))
+ } else if (videoId == "Missing API Key") {
+ onIntentTriggered(DeviceAction.SearchSong(searchQuery))
+ onMissingApiKey(searchQuery)
+ } else {
+ onIntentTriggered(DeviceAction.PlaySong(videoId))
+ onSuccess(
+ searchQuery,
+ videoId,
+ thumbnailUrl,
+ URI("https://www.youtube.com/watch?v=$videoId")
+ )
+ }
+ } catch (e: Exception) {
+ System.err.println("Error playing song: ${e.message}")
+ onFailure(resourceProvider.getString("generic_error"))
+ }
+ }
+
+ private suspend fun youtubeApiCall(query: String): Pair {
+ val deferredResult = CompletableDeferred>()
+
+ val urlBuilder = "https://www.googleapis.com/youtube/v3/search"
+ .toHttpUrlOrNull()
+ ?.newBuilder()
+ ?.addQueryParameter("key", loadYoutubeKey())
+ ?.addQueryParameter("q", query)
+ ?.addQueryParameter("type", "video")
+ ?.addQueryParameter("part", "snippet")
+ ?.addQueryParameter("maxResults", "1")
+ ?.build()
+
+ val request = Request.Builder()
+ .url(urlBuilder.toString())
+ .build()
+
+ client.newCall(request).enqueue(object : Callback {
+ override fun onFailure(call: Call, e: IOException) {
+ deferredResult.completeExceptionally(e)
+ }
+
+ override fun onResponse(call: Call, response: Response) {
+ if (response.isSuccessful) {
+ val body = response.body?.string()
+ if (body != null) {
+ deferredResult.complete(extractVideoIdAndThumbnail(body))
+ } else {
+ deferredResult.complete(Pair("", ""))
+ }
+ } else {
+ if (response.code == 403) {
+ deferredResult.complete(Pair("Missing API Key", ""))
+ } else {
+ deferredResult.complete(Pair("", ""))
+ }
+ }
+ response.close()
+ }
+ })
+
+ return deferredResult.await()
+ }
+
+ private fun extractVideoIdAndThumbnail(jsonString: String): Pair {
+ try {
+ val response = json.decodeFromString(jsonString)
+ val items = response.items ?: return Pair("", "")
+ if (items.isNotEmpty()) {
+ val firstItem = items[0]
+ val idObject = firstItem.id ?: return Pair("", "")
+ if (idObject.kind == "youtube#video") {
+ val videoId = idObject.videoId ?: ""
+ val thumbnailUrl = firstItem.snippet?.thumbnails?.high?.url ?: ""
+ return Pair(videoId, thumbnailUrl)
+ }
+ }
+ } catch (e: Exception) {
+ System.err.println("Error parsing YouTube JSON: ${e.message}")
+ }
+ return Pair("", "")
+ }
+
+ private fun loadYoutubeKey(): String? {
+ var youtubeKey = settingsRepository.getYoutubeApiKey()
+ if (youtubeKey.isNullOrBlank()) {
+ youtubeKey = BuildConfig.YOUTUBE_API_KEY
+ }
+ return youtubeKey
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/ProcessChatCommandUseCase.kt b/app/src/main/java/com/app/assistant/usecase/ProcessChatCommandUseCase.kt
new file mode 100644
index 0000000..bfe96fa
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/ProcessChatCommandUseCase.kt
@@ -0,0 +1,193 @@
+package com.app.assistant.usecase
+
+import com.app.assistant.llm.LlmMessage
+import com.app.assistant.model.Conversation
+import com.app.assistant.repository.SettingsRepository
+import com.app.assistant.util.Category
+import com.google.mediapipe.tasks.text.textclassifier.TextClassifierResult
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import kotlinx.coroutines.flow.flow
+import kotlinx.coroutines.flow.flowOn
+import kotlinx.coroutines.flow.collect
+import java.util.Locale
+
+class ProcessChatCommandUseCase(
+ private val getAiResponseUseCase: GetAiResponseUseCase,
+ private val settingsRepository: SettingsRepository
+) {
+ fun cleanAndPunctuate(input: String): String {
+ val trimmedInput = input.trim()
+ if (trimmedInput.isEmpty()) return ""
+
+ val lastChar = trimmedInput.last()
+ return if (lastChar == '.' || lastChar == '?') {
+ trimmedInput
+ } else {
+ val questionWords = listOf("wh", "how", "can", "do", "is", "are", "does", "did", "will", "could", "should", "would")
+ if (questionWords.any { trimmedInput.lowercase().startsWith(it) }) {
+ "$trimmedInput?"
+ } else {
+ trimmedInput
+ }
+ }
+ }
+
+ fun isNegativeOrNotRequired(phrase: String): Boolean {
+ return negativePatterns.any { it.containsMatchIn(phrase) }
+ }
+
+ fun resolveCategory(results: TextClassifierResult, inputText: String): Category {
+ val categories = results
+ .classificationResult()
+ .classifications()
+ .first()
+ .categories()
+
+
+
+ val highestCategory = categories.maxByOrNull { it.score() }
+ val highestCategoryEnum = highestCategory?.categoryName()?.uppercase()?.let { Category.valueOf(it) }
+
+ val wordsToMatch = categoryWords[highestCategoryEnum] ?: emptyList()
+ val containsWord = wordsToMatch.any { word -> word in inputText.lowercase() }
+
+ val finalCategory = if (wordsToMatch.isNotEmpty() && !containsWord) {
+ categories.firstOrNull { it.categoryName().equals(Category.OTHER.name, ignoreCase = true) }
+ } else {
+ highestCategory
+ }
+
+ return getCategoryFromString(finalCategory?.categoryName()?.uppercase() ?: "")
+ }
+
+ fun getCategoryFromString(categoryName: String): Category =
+ Category.entries.find { it.name == categoryName.uppercase() } ?: Category.OTHER
+
+ suspend fun getAiChatResponse(
+ systemContext: String,
+ chatHistory: List,
+ isHandsFreeActive: Boolean = false,
+ isVisionActive: Boolean = false,
+ isScreenActive: Boolean = false
+ ): String? = withContext(Dispatchers.IO) {
+ val messages = mapConversationsToLlmMessages(systemContext, chatHistory, isHandsFreeActive, isVisionActive, isScreenActive)
+ getAiResponseUseCase.execute(messages)
+ }
+
+ fun getAiChatResponseStream(
+ systemContext: String,
+ chatHistory: List,
+ isHandsFreeActive: Boolean = false,
+ isVisionActive: Boolean = false,
+ isScreenActive: Boolean = false
+ ): kotlinx.coroutines.flow.Flow = flow {
+ val messages = withContext(Dispatchers.IO) {
+ mapConversationsToLlmMessages(systemContext, chatHistory, isHandsFreeActive, isVisionActive, isScreenActive)
+ }
+ getAiResponseUseCase.executeStream(messages).collect {
+ emit(it)
+ }
+ }.flowOn(Dispatchers.IO)
+
+ private fun mapConversationsToLlmMessages(
+ systemContext: String,
+ chatHistory: List,
+ isHandsFreeActive: Boolean = false,
+ isVisionActive: Boolean = false,
+ isScreenActive: Boolean = false
+ ): List {
+ val messages = mutableListOf()
+ messages.add(LlmMessage(role = "system", content = systemContext))
+
+ val latestImageAttachmentId = if (isHandsFreeActive && (isVisionActive || isScreenActive)) {
+ chatHistory.flatMap { it.attachments }
+ .lastOrNull { it.mimeType.startsWith("image/") }
+ ?.id
+ } else {
+ null
+ }
+
+ for (item in chatHistory) {
+ if (item.isLoading) continue
+ val role = if (item.isMe) "user" else "assistant"
+ val llmAttachments = if (item.attachments.isNotEmpty()) {
+ val filtered = item.attachments.filter { att ->
+ if (att.mimeType.startsWith("text/") || att.mimeType == "application/json") {
+ false
+ } else {
+ when {
+ att.mimeType.startsWith("image/") -> {
+ val isSupported = settingsRepository.getIsImageSupported() || att.fileName.startsWith("vision_frame")
+ val isLatest = latestImageAttachmentId == null || att.id == latestImageAttachmentId
+ isSupported && isLatest
+ }
+ att.mimeType.startsWith("audio/") -> settingsRepository.getIsAudioSupported()
+ att.mimeType.startsWith("video/") -> settingsRepository.getIsVideoSupported()
+ att.mimeType.startsWith("application/pdf") -> settingsRepository.getIsDocumentSupported()
+ else -> false
+ }
+ }
+ }
+ if (filtered.isNotEmpty()) {
+ filtered.map { att ->
+ val file = java.io.File(att.filePath)
+ val base64 = if (file.exists()) {
+ val encryptedBytes = file.readBytes()
+ val decryptedBytes = com.app.assistant.db.EncryptionUtil.decryptFile(encryptedBytes, att.iv)
+ android.util.Base64.encodeToString(decryptedBytes, android.util.Base64.NO_WRAP)
+ } else {
+ ""
+ }
+ com.app.assistant.llm.LlmAttachment(
+ base64Data = base64,
+ mimeType = att.mimeType,
+ fileName = att.fileName
+ )
+ }
+ } else null
+ } else null
+ messages.add(LlmMessage(role = role, content = item.text, attachments = llmAttachments))
+ }
+ return messages
+ }
+
+ companion object {
+ private val negativePatterns = listOf(
+ Regex("\\bno\\b", RegexOption.IGNORE_CASE),
+ Regex("\\bnah\\b", RegexOption.IGNORE_CASE),
+ Regex("\\bnot\\b.*", RegexOption.IGNORE_CASE),
+ Regex("\\bnever\\b.*", RegexOption.IGNORE_CASE),
+ Regex("\\bforget\\b.*", RegexOption.IGNORE_CASE),
+ Regex("\\bleave\\b.*", RegexOption.IGNORE_CASE),
+ Regex("\\bdrop\\b.*", RegexOption.IGNORE_CASE),
+ Regex("\\bcancel\\b", RegexOption.IGNORE_CASE),
+ Regex("\\babort\\b", RegexOption.IGNORE_CASE),
+ Regex("\\bstop\\b", RegexOption.IGNORE_CASE),
+ Regex("\\bquit\\b", RegexOption.IGNORE_CASE),
+ Regex("\\bdisregard\\b", RegexOption.IGNORE_CASE),
+ Regex("changed my mind", RegexOption.IGNORE_CASE),
+ Regex("don't bother", RegexOption.IGNORE_CASE),
+ Regex("let it go", RegexOption.IGNORE_CASE),
+ Regex("scratch that", RegexOption.IGNORE_CASE),
+ Regex("just kidding", RegexOption.IGNORE_CASE)
+ )
+
+ private val categoryWords = mapOf(
+ Category.CALL to listOf("call", "phone", "ring", "connect", "need", "get", "dial"),
+ Category.SONGS to listOf("music", "song", "play", "tune", "listen"),
+ Category.ALARM to listOf("alarm", "wake", "set", "remind", "morning"),
+ Category.REMINDER to listOf("remind", "notify", "alert", "remember"),
+ Category.NAVIGATION to listOf(
+ "navigate", "find", "show", "directions", "take me to",
+ "where is", "get to", "go to", "navigation to", "way"
+ ),
+ Category.WEATHER to listOf(
+ "sunny", "cloudy", "rain", "temperature", "umbrella", "weather",
+ "outside", "report", "forecast", "carry", "going to", "how is", "will it be"
+ ),
+ Category.SETTINGS to emptyList(),
+ Category.OTHER to emptyList()
+ )
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/ResourceProvider.kt b/app/src/main/java/com/app/assistant/usecase/ResourceProvider.kt
new file mode 100644
index 0000000..52a3787
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/ResourceProvider.kt
@@ -0,0 +1,5 @@
+package com.app.assistant.usecase
+
+interface ResourceProvider {
+ fun getString(key: String): String
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/SetAlarmUseCase.kt b/app/src/main/java/com/app/assistant/usecase/SetAlarmUseCase.kt
new file mode 100644
index 0000000..3349097
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/SetAlarmUseCase.kt
@@ -0,0 +1,148 @@
+package com.app.assistant.usecase
+
+import com.app.assistant.model.DeviceAction
+import java.util.Calendar
+import java.util.Locale
+
+class SetAlarmUseCase(private val resourceProvider: ResourceProvider) {
+ suspend fun execute(
+ prompt: String,
+ dayOverride: String? = null,
+ onPromptForTime: suspend (dayMatch: String?) -> Unit,
+ onSuccess: suspend (action: DeviceAction) -> Unit,
+ onFailure: suspend (errorMsg: String) -> Unit
+ ) {
+ try {
+ val sanitizedPrompt = prompt.replace(PunctuationRegex, "")
+
+ val dayMatch = dayOverride ?: DayRegex.find(sanitizedPrompt)?.value?.lowercase(Locale.ROOT)
+ val timeMatch = TimeRegex.find(sanitizedPrompt)
+ val relativeTimeMatch = RelativeTimeRegex.find(sanitizedPrompt)
+
+ if (timeMatch == null && relativeTimeMatch == null) {
+ onPromptForTime(dayMatch)
+ } else {
+ val action = calculateAlarmAction(dayMatch, timeMatch, relativeTimeMatch, resourceProvider.getString("new_alarm_message"))
+ onSuccess(action)
+ }
+ } catch (e: Exception) {
+ System.err.println("Error setting alarm: ${e.message}")
+ onFailure(resourceProvider.getString("generic_error"))
+ }
+ }
+
+ private fun calculateAlarmAction(
+ dayMatch: String?,
+ timeMatch: MatchResult?,
+ relativeTimeMatch: MatchResult?,
+ message: String
+ ): DeviceAction {
+ val calendar = Calendar.getInstance()
+
+ if (relativeTimeMatch != null) {
+ val firstValue = relativeTimeMatch.groupValues[1].takeIf { it.isNotEmpty() }?.toIntOrNull() ?: 0
+ val halfValue = if (relativeTimeMatch.groupValues[2] == "half") 0.5 else 0.0
+ val firstUnit = relativeTimeMatch.groupValues[3].lowercase(Locale.ROOT)
+
+ var secondValue = 0
+ var secondUnit = ""
+
+ if (relativeTimeMatch.groupValues[4].isNotEmpty()) {
+ secondValue = relativeTimeMatch.groupValues[4].toInt()
+ secondUnit = relativeTimeMatch.groupValues[5].lowercase(Locale.ROOT)
+ }
+
+ when {
+ firstUnit.contains("hour") || firstUnit.contains("hrs") -> {
+ calendar.add(Calendar.HOUR_OF_DAY, firstValue)
+ if (halfValue > 0) {
+ calendar.add(Calendar.MINUTE, 30)
+ }
+ }
+ firstUnit.contains("min") -> {
+ calendar.add(Calendar.MINUTE, firstValue)
+ }
+ }
+
+ when {
+ secondUnit.contains("hour") || secondUnit.contains("hrs") -> calendar.add(Calendar.HOUR_OF_DAY, secondValue)
+ secondUnit.contains("minute") || secondUnit.contains("minutes") || secondUnit.contains("min") -> calendar.add(Calendar.MINUTE, secondValue)
+ }
+ } else if (timeMatch != null) {
+ val hour = timeMatch.groupValues[1].toInt()
+ val minutes = timeMatch.groupValues[3].toIntOrNull() ?: 0
+ val amPm = timeMatch.groupValues[4].uppercase(Locale.ROOT)
+
+ val hour24 = when {
+ amPm == "PM" && hour != 12 -> hour + 12
+ amPm == "AM" && hour == 12 -> 0
+ else -> hour
+ }
+
+ calendar.set(Calendar.HOUR_OF_DAY, hour24)
+ calendar.set(Calendar.MINUTE, minutes)
+ }
+
+ val repeatDays = mutableListOf()
+
+ when {
+ dayMatch == "today" -> { /* No change needed */ }
+ dayMatch == "tomorrow" -> {
+ calendar.add(Calendar.DAY_OF_MONTH, 1)
+ }
+ dayMatch == "next week" -> {
+ calendar.add(Calendar.DAY_OF_MONTH, 7)
+ }
+ dayMatch == "next weekend" -> {
+ val currentDayOfWeek = calendar.get(Calendar.DAY_OF_WEEK)
+ val daysUntilSaturday = (Calendar.SATURDAY - currentDayOfWeek + 7) % 7
+ calendar.add(Calendar.DAY_OF_MONTH, daysUntilSaturday)
+ }
+ dayMatch != null -> {
+ val dayMap = mapOf(
+ "sunday" to Calendar.SUNDAY,
+ "monday" to Calendar.MONDAY,
+ "tuesday" to Calendar.TUESDAY,
+ "wednesday" to Calendar.WEDNESDAY,
+ "thursday" to Calendar.THURSDAY,
+ "friday" to Calendar.FRIDAY,
+ "saturday" to Calendar.SATURDAY
+ )
+
+ dayMap.entries.forEach { (dayName, dayConstant) ->
+ if (dayMatch.contains(dayName)) {
+ repeatDays.add(dayConstant)
+ }
+ }
+
+ if (repeatDays.isEmpty()) {
+ val targetDay = dayMap.entries.firstOrNull { dayMatch.contains(it.key) }?.value
+ if (targetDay != null) {
+ val currentDayOfWeek = calendar.get(Calendar.DAY_OF_WEEK)
+ val daysToAdd = (targetDay - currentDayOfWeek + 7) % 7
+ calendar.add(Calendar.DAY_OF_MONTH, if (daysToAdd == 0) 7 else daysToAdd)
+ }
+ }
+ }
+ }
+
+ return DeviceAction.SetAlarm(
+ message = message,
+ hour = calendar.get(Calendar.HOUR_OF_DAY),
+ minutes = calendar.get(Calendar.MINUTE),
+ repeatDays = repeatDays,
+ isReminder = false
+ )
+ }
+
+ companion object {
+ private val PunctuationRegex = "[.?!]+".toRegex()
+ private val DayRegex = ("(?i)\\b(today|tomorrow|next week|next weekend|" +
+ "((Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday)" +
+ "( morning| evening| night| afternoon)?))\\b").toRegex(RegexOption.IGNORE_CASE)
+ private val TimeRegex = "(?i)\\b(\\d{1,2})(:(\\d{2}))?\\s*(AM|PM)?\\b".toRegex(RegexOption.IGNORE_CASE)
+ private val RelativeTimeRegex = ("(?i)\\b(\\d+)?\\s*(?:and\\s*(half))?\\s*" +
+ "(hours?|hrs?|mins?|minutes?|seconds?|secs?)\\s*(?:and\\s*(\\d+))?\\s*" +
+ "(mins?|minutes?|secs?|seconds?)?\\s*(from now|later|next)?\\b").toRegex(RegexOption.IGNORE_CASE)
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/usecase/SetReminderUseCase.kt b/app/src/main/java/com/app/assistant/usecase/SetReminderUseCase.kt
new file mode 100644
index 0000000..726e646
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/usecase/SetReminderUseCase.kt
@@ -0,0 +1,153 @@
+package com.app.assistant.usecase
+
+import com.app.assistant.model.DeviceAction
+import java.util.Calendar
+import java.util.Locale
+
+class SetReminderUseCase(private val resourceProvider: ResourceProvider) {
+ suspend fun execute(
+ prompt: String,
+ dayOverride: String? = null,
+ contextOverride: String? = null,
+ onPromptForTime: suspend (dayMatch: String?, context: String) -> Unit,
+ onSuccess: suspend (action: DeviceAction) -> Unit,
+ onFailure: suspend (errorMsg: String) -> Unit
+ ) {
+ try {
+ val sanitizedPrompt = prompt.replace(PunctuationRegex, "")
+
+ val contextMatch = ContextRegex.find(sanitizedPrompt)
+ val context = contextOverride ?: contextMatch?.groupValues?.get(1)?.trim() ?: resourceProvider.getString("reminder_default_context")
+
+ val timeMatch = TimeRegex.find(sanitizedPrompt)
+ val relativeTimeMatch = RelativeTimeRegex.find(sanitizedPrompt)
+ val dayMatch = dayOverride ?: DayRegex.find(sanitizedPrompt)?.value?.lowercase(Locale.ROOT)
+
+ if (timeMatch == null && relativeTimeMatch == null) {
+ onPromptForTime(dayMatch, context)
+ } else {
+ val action = calculateReminderAction(dayMatch, timeMatch, relativeTimeMatch, context)
+ onSuccess(action)
+ }
+ } catch (e: Exception) {
+ System.err.println("Error setting reminder: ${e.message}")
+ onFailure(resourceProvider.getString("generic_error"))
+ }
+ }
+
+ private fun calculateReminderAction(
+ dayMatch: String?,
+ timeMatch: MatchResult?,
+ relativeTimeMatch: MatchResult?,
+ context: String
+ ): DeviceAction {
+ val calendar = Calendar.getInstance()
+
+ if (relativeTimeMatch != null) {
+ val firstValue = relativeTimeMatch.groupValues[1].takeIf { it.isNotEmpty() }?.toIntOrNull() ?: 0
+ val halfValue = if (relativeTimeMatch.groupValues[2] == "half") 0.5 else 0.0
+ val firstUnit = relativeTimeMatch.groupValues[3].lowercase(Locale.ROOT)
+
+ var secondValue = 0
+ var secondUnit = ""
+
+ if (relativeTimeMatch.groupValues[4].isNotEmpty()) {
+ secondValue = relativeTimeMatch.groupValues[4].toInt()
+ secondUnit = relativeTimeMatch.groupValues[5].lowercase(Locale.ROOT)
+ }
+
+ when {
+ firstUnit.contains("hour") || firstUnit.contains("hrs") -> {
+ calendar.add(Calendar.HOUR_OF_DAY, firstValue)
+ if (halfValue > 0) {
+ calendar.add(Calendar.MINUTE, 30)
+ }
+ }
+ firstUnit.contains("min") -> {
+ calendar.add(Calendar.MINUTE, firstValue)
+ }
+ }
+
+ when {
+ secondUnit.contains("hour") || secondUnit.contains("hrs") -> calendar.add(Calendar.HOUR_OF_DAY, secondValue)
+ secondUnit.contains("minute") || secondUnit.contains("minutes") || secondUnit.contains("min") -> calendar.add(Calendar.MINUTE, secondValue)
+ }
+ } else if (timeMatch != null) {
+ val hour = timeMatch.groupValues[1].toInt()
+ val minutes = timeMatch.groupValues[3].toIntOrNull() ?: 0
+ val amPm = timeMatch.groupValues[4].uppercase(Locale.ROOT)
+
+ val hour24 = when {
+ amPm == "PM" && hour != 12 -> hour + 12
+ amPm == "AM" && hour == 12 -> 0
+ else -> hour
+ }
+
+ calendar.set(Calendar.HOUR_OF_DAY, hour24)
+ calendar.set(Calendar.MINUTE, minutes)
+ }
+
+ val repeatDays = mutableListOf()
+
+ when {
+ dayMatch == "today" -> { /* No change needed */ }
+ dayMatch == "tomorrow" -> {
+ calendar.add(Calendar.DAY_OF_MONTH, 1)
+ }
+ dayMatch == "next week" -> {
+ calendar.add(Calendar.DAY_OF_MONTH, 7)
+ }
+ dayMatch == "next weekend" -> {
+ val currentDayOfWeek = calendar.get(Calendar.DAY_OF_WEEK)
+ val daysUntilSaturday = (Calendar.SATURDAY - currentDayOfWeek + 7) % 7
+ calendar.add(Calendar.DAY_OF_MONTH, daysUntilSaturday)
+ }
+ dayMatch != null -> {
+ val dayMap = mapOf(
+ "sunday" to Calendar.SUNDAY,
+ "monday" to Calendar.MONDAY,
+ "tuesday" to Calendar.TUESDAY,
+ "wednesday" to Calendar.WEDNESDAY,
+ "thursday" to Calendar.THURSDAY,
+ "friday" to Calendar.FRIDAY,
+ "saturday" to Calendar.SATURDAY
+ )
+
+ dayMap.entries.forEach { (dayName, dayConstant) ->
+ if (dayMatch.contains(dayName)) {
+ repeatDays.add(dayConstant)
+ }
+ }
+
+ if (repeatDays.isEmpty()) {
+ val targetDay = dayMap.entries.firstOrNull { dayMatch.contains(it.key) }?.value
+ if (targetDay != null) {
+ val currentDayOfWeek = calendar.get(Calendar.DAY_OF_WEEK)
+ val daysToAdd = (targetDay - currentDayOfWeek + 7) % 7
+ calendar.add(Calendar.DAY_OF_MONTH, if (daysToAdd == 0) 7 else daysToAdd)
+ }
+ }
+ }
+ }
+
+ return DeviceAction.SetAlarm(
+ message = context,
+ hour = calendar.get(Calendar.HOUR_OF_DAY),
+ minutes = calendar.get(Calendar.MINUTE),
+ repeatDays = repeatDays,
+ isReminder = true
+ )
+ }
+
+ companion object {
+ private val PunctuationRegex = "[.?!]+".toRegex()
+ private val DayRegex = ("(?i)\\b(today|tomorrow|next week|next weekend|" +
+ "((Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday)" +
+ "( morning| evening| night| afternoon)?))\\b").toRegex(RegexOption.IGNORE_CASE)
+ private val TimeRegex = "(?i)\\b(\\d{1,2})(:(\\d{2}))?\\s*(AM|PM)?\\b".toRegex(RegexOption.IGNORE_CASE)
+ private val RelativeTimeRegex = ("(?i)\\b(\\d+)?\\s*(?:and\\s*(half))?\\s*" +
+ "(hours?|hrs?|mins?|minutes?|seconds?|secs?)\\s*(?:and\\s*(\\d+))?\\s*" +
+ "(mins?|minutes?|secs?|seconds?)?\\s*(from now|later|next)?\\b").toRegex(RegexOption.IGNORE_CASE)
+ private val ContextRegex = "remind me(?: to| for| of| about| to create)?\\s+(.*?)(?:\\s+(at|on|in)\\s+.*)?$".toRegex(RegexOption.IGNORE_CASE)
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/util/Category.kt b/app/src/main/java/com/app/assistant/util/Category.kt
index 06c211c..0198f3f 100644
--- a/app/src/main/java/com/app/assistant/util/Category.kt
+++ b/app/src/main/java/com/app/assistant/util/Category.kt
@@ -1,5 +1,12 @@
package com.app.assistant.util
enum class Category {
- SONGS, CALL, ALARM, REMINDER, OTHER, NAVIGATION, SETTINGS, WEATHER
-}
\ No newline at end of file
+ SONGS,
+ CALL,
+ ALARM,
+ REMINDER,
+ OTHER,
+ NAVIGATION,
+ SETTINGS,
+ WEATHER,
+}
diff --git a/app/src/main/java/com/app/assistant/util/Constants.kt b/app/src/main/java/com/app/assistant/util/Constants.kt
index 1d6d051..8552d88 100644
--- a/app/src/main/java/com/app/assistant/util/Constants.kt
+++ b/app/src/main/java/com/app/assistant/util/Constants.kt
@@ -1,6 +1,12 @@
package com.app.assistant.util
object Constants {
- const val MAIN_CONTEXT = "You are my smart assistant who answers just like normal human response, not too long. Don't refer yourself as an AI."
- val CATEGORY_CONTEXT = "You are a assistant who can classify commands based on previous conversation, from these categories. "+ enumValues().joinToString(",") { it.name } +". Just say the category name."
-}
\ No newline at end of file
+ const val MAIN_CONTEXT =
+ "You are my smart assistant who answers just like normal human response, not too long. Don't refer yourself as an AI."
+ const val HANDS_FREE_MODE_IMAGE_CONTEXT =
+ "You are my smart assistant who answers just like normal human response, not too long. Don't refer yourself as an AI and don't mention words like image, photo, picture, screen in your response. "
+ val CATEGORY_CONTEXT =
+ "You are a assistant who can classify commands based on previous conversation, from these categories. " +
+ enumValues().joinToString(",") { it.name } +
+ ". Just say the category name."
+}
diff --git a/app/src/main/java/com/app/assistant/util/IdGenerator.kt b/app/src/main/java/com/app/assistant/util/IdGenerator.kt
new file mode 100644
index 0000000..bdee17b
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/util/IdGenerator.kt
@@ -0,0 +1,18 @@
+package com.app.assistant.util
+
+import java.util.concurrent.atomic.AtomicLong
+
+object IdGenerator {
+ private val lastId = AtomicLong(0L)
+
+ fun nextId(): Long {
+ while (true) {
+ val now = System.currentTimeMillis()
+ val last = lastId.get()
+ val next = if (now > last) now else last + 1
+ if (lastId.compareAndSet(last, next)) {
+ return next
+ }
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/util/LocaleUtils.kt b/app/src/main/java/com/app/assistant/util/LocaleUtils.kt
new file mode 100644
index 0000000..411e5b1
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/util/LocaleUtils.kt
@@ -0,0 +1,24 @@
+package com.app.assistant.util
+
+import java.util.Locale
+
+object LocaleUtils {
+ fun getLocaleCode(languageCode: String): String {
+ val languageCountryMapping = mapOf(
+ "af" to "ZA", "sq" to "AL", "ar" to "SA", "be" to "BY", "bg" to "BG",
+ "bn" to "BD", "ca" to "ES", "zh" to "CN", "hr" to "HR", "cs" to "CZ",
+ "da" to "DK", "nl" to "NL", "en" to "US", "eo" to "EO", "et" to "EE",
+ "fi" to "FI", "fr" to "FR", "gl" to "ES", "ka" to "GE", "de" to "DE",
+ "el" to "GR", "gu" to "IN", "ht" to "HT", "he" to "IL", "hi" to "IN",
+ "hu" to "HU", "is" to "IS", "id" to "ID", "ga" to "IE", "it" to "IT",
+ "ja" to "JP", "kn" to "IN", "ko" to "KR", "lt" to "LT", "lv" to "LV",
+ "mk" to "MK", "mr" to "IN", "ms" to "MY", "mt" to "MT", "no" to "NO",
+ "fa" to "IR", "pl" to "PL", "pt" to "BR", "ro" to "RO", "ru" to "RU",
+ "sk" to "SK", "sl" to "SI", "es" to "ES", "sv" to "SE", "sw" to "KE",
+ "tl" to "PH", "ta" to "IN", "te" to "IN", "th" to "TH", "tr" to "TR",
+ "uk" to "UA", "ur" to "PK", "vi" to "VN", "cy" to "GB"
+ )
+ val countryCode = languageCountryMapping[languageCode] ?: Locale.getDefault().country
+ return "$languageCode-$countryCode"
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/util/LockState.kt b/app/src/main/java/com/app/assistant/util/LockState.kt
index 631b639..5229706 100644
--- a/app/src/main/java/com/app/assistant/util/LockState.kt
+++ b/app/src/main/java/com/app/assistant/util/LockState.kt
@@ -4,14 +4,19 @@ sealed class LockState {
object None : LockState() // Default state when no task is locked
data class LockAlarm(
- var day: String? = null, // Already processed day (e.g., "tomorrow")
- var time: String? = null // Time to be provided
+ // Already processed day (e.g., "tomorrow")
+ var day: String? = null,
+ // Time to be provided
+ var time: String? = null,
) : LockState()
data class LockReminder(
- var day: String? = null, // Already processed day (e.g., "tomorrow")
- var time: String? = null, // Time to be provided
- var context: String? = null // Message for reminder
+ // Already processed day (e.g., "tomorrow")
+ var day: String? = null,
+ // Time to be provided
+ var time: String? = null,
+ // Message for reminder
+ var context: String? = null,
) : LockState()
object LockNavigation : LockState() // No additional data needed
diff --git a/app/src/main/java/com/app/assistant/util/MarkdownUtils.kt b/app/src/main/java/com/app/assistant/util/MarkdownUtils.kt
new file mode 100644
index 0000000..7641528
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/util/MarkdownUtils.kt
@@ -0,0 +1,250 @@
+package com.app.assistant.util
+
+import androidx.compose.ui.graphics.Color
+import androidx.compose.ui.text.AnnotatedString
+import androidx.compose.ui.text.SpanStyle
+import androidx.compose.ui.text.font.FontFamily
+import androidx.compose.ui.text.font.FontStyle
+import androidx.compose.ui.text.font.FontWeight
+import androidx.compose.ui.text.style.TextDecoration
+import androidx.compose.ui.unit.sp
+import org.commonmark.node.BlockQuote
+import org.commonmark.node.BulletList
+import org.commonmark.node.Code
+import org.commonmark.node.Emphasis
+import org.commonmark.node.FencedCodeBlock
+import org.commonmark.node.HardLineBreak
+import org.commonmark.node.Heading
+import org.commonmark.node.IndentedCodeBlock
+import org.commonmark.node.Link
+import org.commonmark.node.ListItem
+import org.commonmark.node.Node
+import org.commonmark.node.OrderedList
+import org.commonmark.node.Paragraph
+import org.commonmark.node.SoftLineBreak
+import org.commonmark.node.StrongEmphasis
+import org.commonmark.node.Text
+import org.commonmark.node.ThematicBreak
+import org.commonmark.parser.Parser
+
+object MarkdownUtils {
+ private val HeaderRegex = Regex("#+ ")
+ private val BoldRegex = Regex("\\*\\*(.*?)\\*\\*")
+ private val ItalicRegex = Regex("\\*(.*?)\\*")
+ private val ImageRegex = Regex("!\\[.*?\\]\\(.*?\\)")
+ private val LinkRegex = Regex("\\[.*?\\]\\(.*?\\)")
+ private val CodeRegex = Regex("`{1,3}.*?`{1,3}")
+ private val NewlinesRegex = Regex("\\n\\n")
+
+ private val parser = Parser.builder().build()
+
+ fun markdownToPlainText(markdown: String): String {
+ var plainText = markdown
+ plainText = plainText.replace(HeaderRegex, "") // Remove headers
+ plainText = plainText.replace(BoldRegex, "$1") // Bold
+ plainText = plainText.replace(ItalicRegex, "$1") // Italic
+ plainText = plainText.replace(ImageRegex, "") // Images
+ plainText = plainText.replace(LinkRegex, "") // Links
+ plainText = plainText.replace(CodeRegex, "") // Inline code
+ plainText = plainText.replace(NewlinesRegex, "\n") // Simplify newlines
+ return plainText.trim()
+ }
+
+ fun parseMarkdownToAnnotatedString(
+ markdown: String,
+ linkColor: Color = Color.Blue
+ ): AnnotatedString {
+ val document = parser.parse(markdown)
+ val builder = AnnotatedString.Builder()
+ processNodes(document, builder, linkColor)
+ return builder.toAnnotatedString()
+ }
+
+ private fun processNodes(
+ node: Node,
+ builder: AnnotatedString.Builder,
+ linkColor: Color,
+ ) {
+ var child = node.firstChild
+ while (child != null) {
+ val nextSibling = child.next
+ when (child) {
+ is Text -> {
+ builder.append(child.literal)
+ }
+
+ is Emphasis -> {
+ val start = builder.length
+ processNodes(child, builder, linkColor)
+ val end = builder.length
+ builder.addStyle(SpanStyle(fontStyle = FontStyle.Italic), start, end)
+ }
+
+ is StrongEmphasis -> {
+ val start = builder.length
+ processNodes(child, builder, linkColor)
+ val end = builder.length
+ builder.addStyle(SpanStyle(fontWeight = FontWeight.Bold), start, end)
+ }
+
+ is Code -> {
+ // Inline code (single backticks)
+ val start = builder.length
+ builder.append(child.literal)
+ val end = builder.length
+ builder.addStyle(
+ SpanStyle(fontFamily = FontFamily.Monospace),
+ start,
+ end,
+ )
+ }
+
+ is Link -> {
+ val start = builder.length
+ processNodes(child, builder, linkColor)
+ val end = builder.length
+ builder.addStringAnnotation(
+ tag = "URL",
+ annotation = child.destination,
+ start = start,
+ end = end,
+ )
+ builder.addStyle(
+ SpanStyle(color = linkColor, textDecoration = TextDecoration.Underline),
+ start,
+ end,
+ )
+ }
+
+ is Paragraph -> {
+ processNodes(child, builder, linkColor)
+ if (nextSibling != null) {
+ builder.append("\n")
+ }
+ }
+
+ is Heading -> {
+ if (builder.isNotEmpty() && !builder.toString().endsWith("\n")) {
+ builder.append("\n")
+ }
+ val start = builder.length
+ processNodes(child, builder, linkColor)
+ val end = builder.length
+ val headingStyle =
+ when (child.level) {
+ 1 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 28.sp)
+ 2 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 24.sp)
+ 3 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 20.sp)
+ 4 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 18.sp)
+ 5 -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 16.sp)
+ else -> SpanStyle(fontWeight = FontWeight.Bold, fontSize = 14.sp)
+ }
+ builder.addStyle(headingStyle, start, end)
+ if (nextSibling != null) {
+ builder.append("\n")
+ }
+ }
+
+ is BlockQuote -> {
+ val start = builder.length
+ processNodes(child, builder, linkColor)
+ val end = builder.length
+ builder.addStyle(
+ SpanStyle(color = Color.Gray, fontStyle = FontStyle.Italic),
+ start,
+ end,
+ )
+ if (nextSibling != null) {
+ builder.append("\n")
+ }
+ }
+
+ is BulletList -> {
+ var listItem = child.firstChild
+ while (listItem != null) {
+ if (listItem is ListItem) {
+ builder.append("• ")
+ processNodes(listItem, builder, linkColor)
+ builder.append("\n")
+ }
+ listItem = listItem.next
+ }
+ if (nextSibling != null) {
+ builder.append("\n")
+ }
+ }
+
+ is OrderedList -> {
+ var index = child.startNumber
+ var listItem = child.firstChild
+ while (listItem != null) {
+ if (listItem is ListItem) {
+ builder.append("$index. ")
+ processNodes(listItem, builder, linkColor)
+ builder.append("\n")
+ index++
+ }
+ listItem = listItem.next
+ }
+ if (nextSibling != null) {
+ builder.append("\n")
+ }
+ }
+
+ is FencedCodeBlock -> {
+ val start = builder.length
+ builder.append(child.literal)
+ val end = builder.length
+ builder.addStyle(
+ SpanStyle(
+ fontFamily = FontFamily.Monospace,
+ ),
+ start,
+ end,
+ )
+ if (nextSibling != null) {
+ builder.append("\n")
+ }
+ }
+
+ is IndentedCodeBlock -> {
+ val start = builder.length
+ builder.append(child.literal)
+ val end = builder.length
+ builder.addStyle(
+ SpanStyle(
+ fontFamily = FontFamily.Monospace,
+ ),
+ start,
+ end,
+ )
+ if (nextSibling != null) {
+ builder.append("\n")
+ }
+ }
+
+ is ThematicBreak -> {
+ builder.append("\n----------------\n")
+ if (nextSibling != null) {
+ builder.append("\n")
+ }
+ }
+
+ is SoftLineBreak -> {
+ builder.append(" ")
+ }
+
+ is HardLineBreak -> {
+ builder.append("\n")
+ }
+
+ else -> {
+ processNodes(child, builder, linkColor)
+ }
+ }
+ child = nextSibling
+ }
+ }
+
+ private fun AnnotatedString.Builder.isNotEmpty(): Boolean = this.length > 0
+}
diff --git a/app/src/main/java/com/app/assistant/viewmodel/AndroidDeviceProviders.kt b/app/src/main/java/com/app/assistant/viewmodel/AndroidDeviceProviders.kt
new file mode 100644
index 0000000..bed3d35
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/viewmodel/AndroidDeviceProviders.kt
@@ -0,0 +1,68 @@
+package com.app.assistant.viewmodel
+
+import android.content.Context
+import android.content.Intent
+import android.content.pm.PackageManager
+import android.net.Uri
+import android.provider.AlarmClock
+import androidx.core.content.ContextCompat
+import com.app.assistant.R
+import com.app.assistant.model.DeviceAction
+import com.app.assistant.usecase.PermissionChecker
+import com.app.assistant.usecase.ResourceProvider
+
+class AndroidPermissionChecker(private val context: Context) : PermissionChecker {
+ override fun hasPermission(permission: String): Boolean {
+ return ContextCompat.checkSelfPermission(context, permission) == PackageManager.PERMISSION_GRANTED
+ }
+}
+
+class AndroidResourceProvider(private val context: Context) : ResourceProvider {
+ override fun getString(key: String): String {
+ return when (key) {
+ "song_not_found" -> context.getString(R.string.song_not_found_fallback)
+ "generic_error" -> context.getString(R.string.generic_error_fallback)
+ "location_not_found" -> context.getString(R.string.location_not_found_fallback)
+ "new_alarm_message" -> context.getString(R.string.new_alarm_message)
+ "reminder_default_context" -> context.getString(R.string.reminder_default_context)
+ else -> ""
+ }
+ }
+}
+
+fun DeviceAction.toIntent(): Intent {
+ return when (this) {
+ is DeviceAction.MakeCall -> Intent(Intent.ACTION_CALL).apply {
+ addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
+ data = Uri.parse("tel:$phoneNumber")
+ }
+ is DeviceAction.NavigateTo -> {
+ val gmmIntentUri = Uri.parse("google.navigation:q=$location")
+ Intent(Intent.ACTION_VIEW, gmmIntentUri).apply {
+ addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
+ setPackage("com.google.android.apps.maps")
+ }
+ }
+ is DeviceAction.PlaySong -> Intent(Intent.ACTION_VIEW, Uri.parse("https://www.youtube.com/watch?v=$videoId")).apply {
+ addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
+ }
+ is DeviceAction.SearchSong -> Intent(Intent.ACTION_SEARCH).apply {
+ setPackage("com.google.android.youtube")
+ putExtra("query", query)
+ addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
+ }
+ is DeviceAction.SetAlarm -> Intent(AlarmClock.ACTION_SET_ALARM).apply {
+ putExtra(AlarmClock.EXTRA_MESSAGE, message)
+ putExtra(AlarmClock.EXTRA_HOUR, hour)
+ putExtra(AlarmClock.EXTRA_MINUTES, minutes)
+ putExtra(AlarmClock.EXTRA_SKIP_UI, true)
+ if (isReminder) {
+ putExtra(AlarmClock.EXTRA_RINGTONE, AlarmClock.VALUE_RINGTONE_SILENT)
+ }
+ if (repeatDays.isNotEmpty()) {
+ putExtra(AlarmClock.EXTRA_DAYS, ArrayList(repeatDays))
+ }
+ addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
+ }
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/viewmodel/MainViewModel.kt b/app/src/main/java/com/app/assistant/viewmodel/MainViewModel.kt
index 4cb4122..cd18702 100644
--- a/app/src/main/java/com/app/assistant/viewmodel/MainViewModel.kt
+++ b/app/src/main/java/com/app/assistant/viewmodel/MainViewModel.kt
@@ -1,169 +1,400 @@
package com.app.assistant.viewmodel
-import android.Manifest
-import android.Manifest.permission.CALL_PHONE
-import android.Manifest.permission.READ_CONTACTS
-import android.Manifest.permission.RECORD_AUDIO
-import android.annotation.SuppressLint
-import android.app.Activity
import android.app.Application
-import android.content.ComponentCallbacks2
-import android.content.Context
-import android.content.Intent
-import android.content.IntentSender
-import android.content.SharedPreferences
-import android.content.pm.PackageManager
-import android.content.res.Configuration
-import android.location.Geocoder
-import android.location.LocationManager
-import android.media.AudioManager
-import android.media.MediaPlayer
-import android.net.Uri
-import android.os.Bundle
-import android.os.Handler
-import android.os.Looper
-import android.provider.AlarmClock
-import android.provider.ContactsContract
-import android.speech.RecognitionListener
-import android.speech.RecognizerIntent
-import android.speech.SpeechRecognizer
-import android.speech.tts.TextToSpeech
-import android.speech.tts.UtteranceProgressListener
import android.util.Log
import androidx.compose.runtime.mutableStateOf
import androidx.compose.ui.focus.FocusManager
import androidx.compose.ui.platform.SoftwareKeyboardController
-import androidx.core.app.ActivityCompat
-import androidx.core.content.ContextCompat
import androidx.lifecycle.AndroidViewModel
import androidx.lifecycle.viewModelScope
-import com.app.assistant.BuildConfig
-import com.app.assistant.Classifier.TextClassifierHelper
import com.app.assistant.db.DynamicConversationRepository
import com.app.assistant.db.SyncStateList
-import com.app.assistant.model.Contact
import com.app.assistant.model.Conversation
import com.app.assistant.model.Group
-import com.app.assistant.translation.TranslatorManager
+import com.app.assistant.model.Attachment
+import com.app.assistant.repository.SettingsRepository
+import com.app.assistant.usecase.CallContactUseCase
+import com.app.assistant.usecase.GetWeatherUseCase
+import com.app.assistant.usecase.NavigateUseCase
+import com.app.assistant.usecase.PlaySongUseCase
+import com.app.assistant.usecase.ProcessChatCommandUseCase
+import com.app.assistant.usecase.SetAlarmUseCase
+import com.app.assistant.usecase.SetReminderUseCase
import com.app.assistant.util.Category
import com.app.assistant.util.Constants.MAIN_CONTEXT
+import com.app.assistant.util.Constants.HANDS_FREE_MODE_IMAGE_CONTEXT
import com.app.assistant.util.LockState
-import com.google.android.gms.common.api.ResolvableApiException
-import com.google.android.gms.location.LocationAvailability
-import com.google.android.gms.location.LocationCallback
-import com.google.android.gms.location.LocationRequest
-import com.google.android.gms.location.LocationResult
-import com.google.android.gms.location.LocationServices
-import com.google.android.gms.location.LocationSettingsRequest
-import com.google.android.gms.location.LocationSettingsResponse
-import com.google.android.gms.location.Priority
-import com.google.android.gms.location.SettingsClient
-import com.google.android.gms.tasks.Task
+import com.app.assistant.classifier.TextClassifierHelper
import com.google.mediapipe.tasks.text.textclassifier.TextClassifierResult
-import com.google.mlkit.nl.translate.TranslateLanguage
-import kotlinx.coroutines.CompletableDeferred
-import kotlinx.coroutines.CoroutineScope
-import kotlinx.coroutines.Dispatchers
-import kotlinx.coroutines.delay
+import com.app.assistant.R
import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.MutableStateFlow
+import kotlinx.coroutines.flow.SharedFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asSharedFlow
import kotlinx.coroutines.flow.asStateFlow
+import kotlinx.coroutines.flow.collect
+import kotlinx.coroutines.flow.Flow
+import kotlinx.coroutines.flow.receiveAsFlow
+import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.launch
+import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
-import okhttp3.Call
-import okhttp3.Callback
-import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
-import okhttp3.MediaType.Companion.toMediaTypeOrNull
-import okhttp3.OkHttpClient
-import okhttp3.Request
-import okhttp3.RequestBody.Companion.toRequestBody
-import okhttp3.Response
-import org.json.JSONArray
-import org.json.JSONException
-import org.json.JSONObject
-import java.io.IOException
import java.lang.reflect.Modifier
import java.net.URI
-import java.net.URLEncoder
-import java.net.UnknownHostException
-import java.nio.charset.StandardCharsets
-import java.text.SimpleDateFormat
-import java.util.Calendar
-import java.util.Locale
-import java.util.concurrent.TimeUnit
import kotlin.coroutines.resume
import kotlin.coroutines.suspendCoroutine
-import kotlin.math.max
-import androidx.core.net.toUri
-import androidx.security.crypto.EncryptedSharedPreferences
-import androidx.security.crypto.MasterKeys
+class MainViewModel(
+ application: Application,
+ private val speak: Boolean,
+ internal val settingsRepository: SettingsRepository,
+ private val repository: DynamicConversationRepository,
+ internal val callContactUseCase: CallContactUseCase,
+ internal val playSongUseCase: PlaySongUseCase,
+ internal val navigateUseCase: NavigateUseCase,
+ internal val getWeatherUseCase: GetWeatherUseCase,
+ internal val setAlarmUseCase: SetAlarmUseCase,
+ internal val setReminderUseCase: SetReminderUseCase,
+ internal val processChatCommandUseCase: ProcessChatCommandUseCase,
+) : AndroidViewModel(application) {
+ private val _question = MutableStateFlow("")
+ val question: StateFlow = _question.asStateFlow()
-class MainViewModel(application: Application, private val speak: Boolean) : AndroidViewModel(application) {
- private var globalContext: Context? = application
- private val sharedPreferences: SharedPreferences =
- application.getSharedPreferences("app_prefs", Application.MODE_PRIVATE)
- var question = mutableStateOf("")
- // DB reference
- private val repository = DynamicConversationRepository(application)
- var chatList = SyncStateList(repository)
- private set
+ private val _selectedAttachments = MutableStateFlow>(emptyList())
+ val selectedAttachments: StateFlow> = _selectedAttachments.asStateFlow()
+
+ private fun compressImageIfNeeded(context: android.content.Context, uri: android.net.Uri): ByteArray? {
+ val mimeType = context.contentResolver.getType(uri) ?: return null
+ if (!mimeType.startsWith("image/")) {
+ return null
+ }
+ try {
+ // First read EXIF rotation
+ var rotation = 0
+ try {
+ context.contentResolver.openInputStream(uri)?.use { stream ->
+ val exif = android.media.ExifInterface(stream)
+ val orientation = exif.getAttributeInt(
+ android.media.ExifInterface.TAG_ORIENTATION,
+ android.media.ExifInterface.ORIENTATION_NORMAL
+ )
+ rotation = when (orientation) {
+ android.media.ExifInterface.ORIENTATION_ROTATE_90 -> 90
+ android.media.ExifInterface.ORIENTATION_ROTATE_180 -> 180
+ android.media.ExifInterface.ORIENTATION_ROTATE_270 -> 270
+ else -> 0
+ }
+ }
+ } catch (e: Exception) {
+ Log.e("MainViewModel", "Failed to parse EXIF orientation", e)
+ }
+
+ // Decode dimensions only
+ val options = android.graphics.BitmapFactory.Options().apply {
+ inJustDecodeBounds = true
+ }
+ context.contentResolver.openInputStream(uri)?.use { stream ->
+ android.graphics.BitmapFactory.decodeStream(stream, null, options)
+ }
+
+ val width = options.outWidth
+ val height = options.outHeight
+ if (width <= 0 || height <= 0) return null
+
+ // Calculate sample size (approximate scaling down during decode)
+ val maxDim = 1200
+ var inSampleSize = 1
+ if (width > maxDim || height > maxDim) {
+ val halfWidth = width / 2
+ val halfHeight = height / 2
+ while ((halfWidth / inSampleSize) >= maxDim || (halfHeight / inSampleSize) >= maxDim) {
+ inSampleSize *= 2
+ }
+ }
+
+ // Decode full image with sample size
+ val decodeOptions = android.graphics.BitmapFactory.Options().apply {
+ this.inSampleSize = inSampleSize
+ }
+ val bitmap = context.contentResolver.openInputStream(uri)?.use { stream ->
+ android.graphics.BitmapFactory.decodeStream(stream, null, decodeOptions)
+ } ?: return null
+
+ // Precise scaling to max 1200px
+ val currentWidth = bitmap.width
+ val currentHeight = bitmap.height
+ val scaledBitmap = if (currentWidth > maxDim || currentHeight > maxDim) {
+ val ratio = currentWidth.toFloat() / currentHeight.toFloat()
+ val (newWidth, newHeight) = if (currentWidth > currentHeight) {
+ maxDim to (maxDim / ratio).toInt()
+ } else {
+ (maxDim * ratio).toInt() to maxDim
+ }
+ android.graphics.Bitmap.createScaledBitmap(bitmap, newWidth, newHeight, true).also {
+ if (it != bitmap) {
+ bitmap.recycle()
+ }
+ }
+ } else {
+ bitmap
+ }
+
+ // Correct EXIF rotation
+ val rotatedBitmap = if (rotation != 0) {
+ val matrix = android.graphics.Matrix().apply { postRotate(rotation.toFloat()) }
+ android.graphics.Bitmap.createBitmap(
+ scaledBitmap, 0, 0, scaledBitmap.width, scaledBitmap.height, matrix, true
+ ).also {
+ if (it != scaledBitmap) {
+ scaledBitmap.recycle()
+ }
+ }
+ } else {
+ scaledBitmap
+ }
+
+ // Compress to JPEG 80%
+ val outputStream = java.io.ByteArrayOutputStream()
+ rotatedBitmap.compress(android.graphics.Bitmap.CompressFormat.JPEG, 80, outputStream)
+ val compressedBytes = outputStream.toByteArray()
+ rotatedBitmap.recycle()
+ return compressedBytes
+ } catch (e: Exception) {
+ Log.e("MainViewModel", "Error compressing image", e)
+ return null
+ }
+ }
+
+ fun addSelectedAttachment(uri: android.net.Uri) {
+ viewModelScope.launch(Dispatchers.IO) {
+ val context = getApplication().applicationContext
+ val contentResolver = context.contentResolver
+ var fileName = "file_${System.currentTimeMillis()}"
+ var mimeType = contentResolver.getType(uri) ?: "*/*"
+ contentResolver.query(uri, null, null, null, null)?.use { cursor ->
+ val nameIndex = cursor.getColumnIndex(android.provider.OpenableColumns.DISPLAY_NAME)
+ if (nameIndex != -1 && cursor.moveToFirst()) {
+ fileName = cursor.getString(nameIndex)
+ }
+ }
+ try {
+ val isImage = mimeType.startsWith("image/") ||
+ fileName.endsWith(".jpg", ignoreCase = true) ||
+ fileName.endsWith(".jpeg", ignoreCase = true) ||
+ fileName.endsWith(".png", ignoreCase = true) ||
+ fileName.endsWith(".webp", ignoreCase = true)
+
+ var fileBytes: ByteArray? = null
+ if (isImage) {
+ fileBytes = compressImageIfNeeded(context, uri)
+ if (fileBytes != null) {
+ mimeType = "image/jpeg"
+ if (!fileName.lowercase().endsWith(".jpg") && !fileName.lowercase().endsWith(".jpeg")) {
+ val dotIndex = fileName.lastIndexOf('.')
+ fileName = if (dotIndex != -1) {
+ "${fileName.substring(0, dotIndex)}.jpg"
+ } else {
+ "$fileName.jpg"
+ }
+ }
+ }
+ }
+
+ if (fileBytes == null) {
+ val inputStream = contentResolver.openInputStream(uri) ?: return@launch
+ fileBytes = inputStream.readBytes()
+ inputStream.close()
+ }
+
+ val iv = com.app.assistant.db.EncryptionUtil.generateIV()
+ val encryptedBytes = com.app.assistant.db.EncryptionUtil.encryptFile(fileBytes, iv)
+
+ val attachmentsDir = java.io.File(context.filesDir, "attachments")
+ if (!attachmentsDir.exists()) {
+ attachmentsDir.mkdirs()
+ }
+
+ val targetFile = java.io.File(attachmentsDir, "file_${System.currentTimeMillis()}_${java.util.UUID.randomUUID()}.enc")
+ targetFile.writeBytes(encryptedBytes)
+
+ val attachment = Attachment(
+ filePath = targetFile.absolutePath,
+ mimeType = mimeType,
+ fileName = fileName,
+ iv = iv
+ )
+ _selectedAttachments.value = _selectedAttachments.value + attachment
+ } catch (e: Exception) {
+ Log.e("MainViewModel", "Failed to add attachment", e)
+ }
+ }
+ }
+
+ fun removeSelectedAttachment(attachment: Attachment) {
+ viewModelScope.launch(Dispatchers.IO) {
+ try {
+ val file = java.io.File(attachment.filePath)
+ if (file.exists()) {
+ file.delete()
+ }
+ } catch (e: Exception) {
+ e.printStackTrace()
+ }
+ _selectedAttachments.value = _selectedAttachments.value - attachment
+ }
+ }
+
+ fun setQuestion(text: String) {
+ _question.value = text
+ }
+
+ val chatList: SyncStateList by lazy {
+ SyncStateList(repository, viewModelScope)
+ }
var currentGroupId: Long = -1L
private set
- var languages : List>
- var showBottomSheet = mutableStateOf(false)
- private set
+ private val _showBottomSheet = MutableStateFlow(false)
+ val showBottomSheet: StateFlow = _showBottomSheet.asStateFlow()
+
+ fun setShowBottomSheet(show: Boolean) {
+ _showBottomSheet.value = show
+ }
private val _isLanguageLoading = MutableStateFlow(false)
val isLanguageLoading: StateFlow = _isLanguageLoading
private val _showToastEvent = MutableSharedFlow()
val showToastEvent = _showToastEvent.asSharedFlow()
- private val translatorManager = TranslatorManager()
- //For custom ui
+ // For custom ui
private val _isCustomUI = MutableStateFlow(false)
val isCustomUI: StateFlow = _isCustomUI
- //For custom ui half page
+ // For custom ui half page
private val _isCustomUIHalfPage = MutableStateFlow(false)
- val isCustomUIHalfPage : StateFlow = _isCustomUIHalfPage
+ val isCustomUIHalfPage: StateFlow = _isCustomUIHalfPage
- // MutableStateFlow for isTranslationEnabled
- private val _isTranslationEnabled = MutableStateFlow(
- sharedPreferences.getBoolean("is_translation_enabled", false)
- )
- val isTranslationEnabled: StateFlow = _isTranslationEnabled
- // MutableStateFlow for activeLanguageCode
- private val _activeLanguageCode = MutableStateFlow(
- sharedPreferences.getString("active_language_code", "en") ?: "en"
- )
private var _isSpeaking = MutableStateFlow(false)
val isSpeaking: StateFlow = _isSpeaking
private val _isListening = MutableStateFlow(false)
val isListening: StateFlow = _isListening.asStateFlow()
+ private val _audioAmplitude = MutableStateFlow(0f)
+ val audioAmplitude: StateFlow = _audioAmplitude.asStateFlow()
+
+ fun setAudioAmplitude(amplitude: Float) {
+ _audioAmplitude.value = amplitude
+ }
+
+ private val _isVoiceProcessing = MutableStateFlow(false)
+ val isVoiceProcessing: StateFlow = _isVoiceProcessing.asStateFlow()
+
+ private val _isHandsFreeModeActive = MutableStateFlow(false)
+ val isHandsFreeModeActive: StateFlow = _isHandsFreeModeActive.asStateFlow()
+
+ private val _isVisionModeActive = MutableStateFlow(false)
+ val isVisionModeActive: StateFlow = _isVisionModeActive.asStateFlow()
+
+ fun toggleVisionMode() {
+ val active = !_isVisionModeActive.value
+ _isVisionModeActive.value = active
+ if (active) {
+ _isScreenModeActive.value = false
+ }
+ com.app.assistant.camera.VisualBufferManager.clear()
+ }
+
+ private val _isScreenModeActive = MutableStateFlow(false)
+ val isScreenModeActive: StateFlow = _isScreenModeActive.asStateFlow()
+
+ fun toggleScreenMode() {
+ val active = !_isScreenModeActive.value
+ _isScreenModeActive.value = active
+ if (active) {
+ _isVisionModeActive.value = false
+ }
+ com.app.assistant.camera.VisualBufferManager.clear()
+ }
+
+ fun setScreenModeActive(active: Boolean) {
+ _isScreenModeActive.value = active
+ if (active) {
+ _isVisionModeActive.value = false
+ }
+ com.app.assistant.camera.VisualBufferManager.clear()
+ }
+
+ private val _isMicReady = MutableStateFlow(false)
+ val isMicReady: StateFlow = _isMicReady.asStateFlow()
+
+ private val _isMicMuted = MutableStateFlow(false)
+ val isMicMuted: StateFlow = _isMicMuted.asStateFlow()
+
+ fun toggleMicMute() {
+ val newMuteState = !_isMicMuted.value
+ _isMicMuted.value = newMuteState
+ if (newMuteState) {
+ _isListening.value = false
+ _isMicReady.value = false
+ }
+ }
+
+ fun setMicReady(ready: Boolean) {
+ _isMicReady.value = ready
+ }
+
+ fun toggleHandsFreeMode() {
+ val modelManager = com.app.assistant.speech.SpeechModelManager(getApplication())
+ val isDownloaded = modelManager.isModelDownloaded()
+ if (!isDownloaded && !_isHandsFreeModeActive.value) {
+ triggerToast("Offline speech models are required for Hands-Free mode. Please download them in settings.")
+ return
+ }
+ setHandsFreeModeActive(!_isHandsFreeModeActive.value)
+ }
+
+ fun setHandsFreeModeActive(active: Boolean) {
+ if (active) {
+ val modelManager = com.app.assistant.speech.SpeechModelManager(getApplication())
+ if (!modelManager.isModelDownloaded()) {
+ triggerToast("Offline speech models are required for Hands-Free mode. Please download them in settings.")
+ return
+ }
+ }
+ _isHandsFreeModeActive.value = active
+ if (active) {
+ _isCustomUI.value = false // Keep standard layout size so our overlay takes full screen
+ _isCustomUIHalfPage.value = false // Clear half-page voice panel flags
+ _isListening.value = false
+ _isVoiceProcessing.value = false
+ _isMicReady.value = false
+ } else {
+ stopTextToSpeech()
+ _isListening.value = false
+ _isVoiceProcessing.value = false
+ _isMicReady.value = false
+ _isMicMuted.value = false // Reset mic mute state when exiting Hands-Free mode
+ _isVisionModeActive.value = false // Deactivate vision mode when exiting Hands-Free mode
+ _isScreenModeActive.value = false // Deactivate screen mode when exiting Hands-Free mode
+ }
+ }
+
private val _groupList = MutableStateFlow>(emptyList())
val groupList: StateFlow> = _groupList.asStateFlow()
- private val securedPreferences = EncryptedSharedPreferences.create(
- "secure_prefs", // A name for the preference file
- MasterKeys.getOrCreate(MasterKeys.AES256_GCM_SPEC), // Master key for encryption
- application.applicationContext,
- EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
- EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM
- )
+ private val _isDeletingChat = MutableStateFlow(false)
+ val isDeletingChat: StateFlow = _isDeletingChat.asStateFlow()
- // Text-to-Speech and Translators
- lateinit var textToSpeech: TextToSpeech
+ // UI Event flow
+ internal val _uiEvent = Channel(Channel.BUFFERED)
+ val uiEvent: Flow = _uiEvent.receiveAsFlow()
- //Classifier
+ // Classifier
private lateinit var classifierHelper: TextClassifierHelper
- //LockState
+ // LockState
var lockState: LockState = LockState.None
- set(value) {
+ internal set(value) {
// Automatically clear any stored data when lockState is set to None
if (value is LockState.None) {
clearLockStateData()
@@ -172,54 +403,75 @@ class MainViewModel(application: Application, private val speak: Boolean) : Andr
}
// Classifier listener
- private val listener = object :
- TextClassifierHelper.TextResultsListener {
+ private val listener = object : TextClassifierHelper.TextResultsListener {
override fun onResult(
results: TextClassifierResult,
inferenceTime: Long,
inputText: String,
itemId: Long,
loadingItemId: Long,
- speak: Boolean
+ speak: Boolean,
) {
processClassifierResponse(inferenceTime, results, inputText, itemId, loadingItemId, speak)
}
override fun onError(error: String) {
- Log.d("Classifier error", "Unable to classify$error")
+ Log.e("Classifier error", "Unable to classify: $error")
+ }
+
+ override fun onError(
+ error: String,
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean
+ ) {
+ Log.e("Classifier error", "Unable to classify: $error")
+ viewModelScope.launch(Dispatchers.Main) {
+ val index = chatList.indexOfFirst { it.id == loadingItemId }
+ if (index != -1) {
+ val item = chatList[index].copy(
+ text = "Classification failed to load.",
+ isMe = false,
+ isLoading = false,
+ isStreaming = false,
+ category = Category.OTHER.name
+ )
+ chatList.set(index, item)
+ }
+ }
}
}
init {
- if(speak) {
+ if (speak) {
_isCustomUI.value = true
_isCustomUIHalfPage.value = true
_isListening.value = true
}
- initializeTextToSpeech()
- languages = getPublicStaticFinalStringsWithNames(TranslateLanguage::class.java)
- if(speak)
+ if (speak) {
startSpeechRecognition()
+ }
initializeTextClassifier()
- initializeTranslator();
- loadGroup();
+ loadGroup()
}
- fun expandToFullScreen(){
- if(_isCustomUI.value)
+ fun expandToFullScreen() {
+ if (_isCustomUI.value) {
_isCustomUI.value = false
- if(_isCustomUIHalfPage.value)
+ }
+ if (_isCustomUIHalfPage.value) {
_isCustomUIHalfPage.value = false
+ }
}
- private fun loadGroup(){
+ private fun loadGroup() {
viewModelScope.launch {
_groupList.value = repository.loadAllGroups().toList()
}
}
- fun loadMessagesFromGroup(groupId: Long){
- viewModelScope.launch{
+ fun loadMessagesFromGroup(groupId: Long) {
+ viewModelScope.launch {
val newMessages = repository.loadMessagesForGroup(groupId)
chatList.clear()
chatList.addAll(newMessages)
@@ -227,139 +479,90 @@ class MainViewModel(application: Application, private val speak: Boolean) : Andr
}
}
- private fun initializeTranslator(){
- if(getIsTranslationEnabled() && getActiveLanguageCode() != ""){
- setupTranslator(getActiveLanguageCode())
- }
- }
-
- private fun initializeTextClassifier(){
- // Create the classification helper that will do the heavy lifting
+ private fun initializeTextClassifier() {
classifierHelper = TextClassifierHelper(
context = getApplication().applicationContext,
- listener = listener
+ listener = listener,
)
- classifierHelper.initClassifier()
}
- // Function to update isTranslationEnabled and persist the value
- fun updateTranslationEnabled(enabled: Boolean) {
- _isTranslationEnabled.value = enabled
+ fun setSpeaking(speaking: Boolean) {
+ _isSpeaking.value = speaking
+ }
+
+ fun setListening(listening: Boolean) {
+ _isListening.value = listening
+ }
+
+ fun setVoiceProcessing(processing: Boolean) {
+ _isVoiceProcessing.value = processing
+ }
+
+ fun stopTextToSpeech() {
viewModelScope.launch {
- sharedPreferences.edit().putBoolean("is_translation_enabled", enabled).apply()
- }
- }
-
- // Function to get the current value of isTranslationEnabled
- fun getIsTranslationEnabled(): Boolean {
- return _isTranslationEnabled.value
- }
-
- // Function to update activeLanguageCode and persist the value
- fun updateActiveLanguageCode(languageCode: String) {
- _activeLanguageCode.value = languageCode
- viewModelScope.launch {
- sharedPreferences.edit().putString("active_language_code", languageCode).apply()
- }
- }
-
- // Function to get the current value of ActiveLanguageCode
- fun getActiveLanguageCode(): String {
- return _activeLanguageCode.value
- }
-
- fun stopTextToSpeech(){
- textToSpeech.stop()
- _isSpeaking.value = false;
- }
-
- private fun getPublicStaticFinalStringsWithNames(clazz: Class<*>): List> {
- val publicStaticFinalStringsWithNames = mutableListOf>()
-
- val fields = clazz.declaredFields
- for (field in fields) {
- val modifiers = field.modifiers
- if (Modifier.isPublic(modifiers) && Modifier.isStatic(modifiers) && Modifier.isFinal(modifiers)) {
- if (field.type == String::class.java) {
- try {
- val name = field.name
- val value = field.get(null) as String
- publicStaticFinalStringsWithNames.add(Pair(name, value))
- } catch (e: IllegalAccessException) {
- e.printStackTrace()
- }
- }
- }
- }
-
- return publicStaticFinalStringsWithNames
- }
-
- private fun initializeTextToSpeech() {
- textToSpeech = TextToSpeech(getApplication()) { status ->
- if (status == TextToSpeech.SUCCESS) {
- Log.d("TextToSpeech", "Initialization Success")
- } else {
- Log.d("TextToSpeech", "Initialization Failed")
- }
+ _uiEvent.send(UIEvent.StopSpeaking)
}
+ _isSpeaking.value = false
}
fun shutdownResources() {
- textToSpeech.shutdown()
- translatorManager.closeTranslator()
- globalContext = null
+ classifierHelper.shutDown()
}
fun processQuestion(
focusManager: FocusManager? = null,
keyboardController: SoftwareKeyboardController? = null,
- context: Context,
- speak: Boolean = false
+ speak: Boolean = false,
) {
- globalContext = context
val originalQuestion = question.value
+ val currentAttachments = _selectedAttachments.value
+ _selectedAttachments.value = emptyList()
+
focusManager?.clearFocus()
keyboardController?.hide()
- question.value = ""
+ setQuestion("")
viewModelScope.launch {
val itemId: Long
- val newItem: Conversation = if(getIsTranslationEnabled()){
- Conversation(englishText = "", translatedText = originalQuestion, isMe = true)
-
- }else{
- Conversation(englishText = originalQuestion, translatedText = "", isMe = true)
+ val processedQuestion = processChatCommandUseCase.cleanAndPunctuate(originalQuestion)
+ val finalQuestionText = withContext(Dispatchers.IO) {
+ val finalQuestionBuilder = java.lang.StringBuilder(processedQuestion)
+ currentAttachments.forEach { att ->
+ if (att.mimeType.startsWith("text/") || att.mimeType == "application/json") {
+ try {
+ val file = java.io.File(att.filePath)
+ if (file.exists()) {
+ val encryptedBytes = file.readBytes()
+ val decryptedBytes = com.app.assistant.db.EncryptionUtil.decryptFile(encryptedBytes, att.iv)
+ val textContent = String(decryptedBytes, Charsets.UTF_8)
+ finalQuestionBuilder.append("\n\n[Attached File: ${att.fileName}]\n---\n$textContent\n---\n")
+ }
+ } catch (e: java.lang.Exception) {
+ Log.e("MainViewModel", "Failed to extract text file content", e)
+ }
+ }
+ }
+ finalQuestionBuilder.toString()
}
+ val newItem = Conversation(text = finalQuestionText, isMe = true, attachments = currentAttachments)
+
chatList.add(newItem)
- if(_isCustomUIHalfPage.value)
+ if (_isCustomUIHalfPage.value) {
_isCustomUIHalfPage.value = false
+ }
itemId = newItem.id
- val loadingItem = Conversation(englishText = "", translatedText = "", isMe = false, isLoading = true)
+ val loadingItem = Conversation(text = "", isMe = false, isLoading = true)
chatList.add(loadingItem)
val loadingItemId = loadingItem.id
- //_conversationListUI.add(loadingItem)
- val translatedQuestionInEnglish = if (getIsTranslationEnabled()) {
- translatorManager.translateToEnglishSuspend(originalQuestion) ?: originalQuestion
- } else {
- originalQuestion
- }
-
- val processedQuestion = cleanAndPunctuate(translatedQuestionInEnglish)
- chatList.indexOfFirst { it.id == itemId }.takeIf { it != -1 }?.let { index ->
- val updatedItem = chatList[index].copy(englishText = processedQuestion)
- chatList.set(index, updatedItem)
- }
-
- if(lockState != LockState.None && isNegativeOrNotRequired(processedQuestion)){
+ if (lockState != LockState.None && processChatCommandUseCase.isNegativeOrNotRequired(processedQuestion)) {
lockState = LockState.None
classifierHelper.classify(processedQuestion, itemId, loadingItemId, speak)
- }else{
- when(val state = lockState){
+ } else {
+ when (val state = lockState) {
is LockState.LockAlarm -> handleAlarmLockState(itemId, loadingItemId, speak, state)
is LockState.LockReminder -> handleReminderLockState(itemId, loadingItemId, speak, state)
is LockState.LockNavigation -> TODO()
@@ -369,66 +572,38 @@ class MainViewModel(application: Application, private val speak: Boolean) : Andr
}
}
+
+
private fun processClassifierResponse(
inferenceTime: Long,
results: TextClassifierResult,
inputText: String,
itemId: Long,
loadingItemId: Long,
- speak: Boolean
+ speak: Boolean,
) {
- val categories = results.classificationResult()
- .classifications().first()
- .categories()
+ viewModelScope.launch {
+ val finalCategory = processChatCommandUseCase.resolveCategory(results, inputText)
+ callCommand(finalCategory.name, itemId, loadingItemId, speak)
- // Define words for each category using Category enum
- val categoryWords = mapOf(
- Category.CALL to listOf("call", "phone", "ring", "connect", "need", "get", "dial"),
- Category.SONGS to listOf("music", "song", "play", "tune", "listen"),
- Category.ALARM to listOf("alarm", "wake", "set", "remind", "morning"),
- Category.REMINDER to listOf("remind", "notify", "alert", "remember"),
- Category.NAVIGATION to listOf("navigate","find","show","directions","take me to","where is","get to","go to","navigation to","way"),
- Category.WEATHER to listOf("sunny", "cloudy", "rain", "temperature", "umbrella", "weather", "outside", "report", "forecast", "carry", "going to", "how is", "will it be"),
- Category.SETTINGS to emptyList(),
- Category.OTHER to emptyList() // "OTHER" doesn't need specific words
- )
-
- // Find the category with the highest score
- val highestCategory = categories.maxByOrNull { it.score() }
- val highestCategoryEnum = highestCategory?.categoryName()?.uppercase()?.let { Category.valueOf(it) }
-
- // Check if the highest category has specific words to verify
- val wordsToMatch = categoryWords[highestCategoryEnum] ?: emptyList()
- val containsWord = wordsToMatch.any { word -> word in inputText.lowercase() }
-
- // Set final category based on word verification
- val finalCategory = if (wordsToMatch.isNotEmpty() && !containsWord) {
- // Force the category to "OTHER" if no matching words found for the category
- categories.firstOrNull { it.categoryName().equals(Category.OTHER.name, ignoreCase = true) }
- } else {
- // Keep the highest category if matching words found or no words defined
- highestCategory
- }
-
- val categoryFromString = getCategoryFromString(finalCategory?.categoryName()?.uppercase() ?: "").toString()
-
- CallCommand(categoryFromString, itemId, loadingItemId, speak)
-
- if (finalCategory != null) {
- chatList.find{ it.id == itemId }?.category = getCategoryFromString(finalCategory.categoryName()).name
- Log.d("Classifier Inference Result", finalCategory.categoryName())
+ val index = chatList.indexOfFirst { it.id == itemId }
+ if (index != -1) {
+ val updatedItem = chatList[index].copy(category = finalCategory.name)
+ chatList.set(index, updatedItem)
+ }
+ Log.d("Classifier Inference Result", finalCategory.name)
}
}
- private fun CallCommand(
+ private fun callCommand(
categoryFromString: String,
itemId: Long,
loadingItemId: Long,
- speak: Boolean
+ speak: Boolean,
) {
when (categoryFromString) {
Category.CALL.name -> {
- Call(itemId, loadingItemId, speak, Category.CALL)
+ callContact(itemId, loadingItemId, speak, Category.CALL)
}
Category.OTHER.name -> {
@@ -436,134 +611,235 @@ class MainViewModel(application: Application, private val speak: Boolean) : Andr
}
Category.SETTINGS.name -> {
- // TODO
callAI(loadingItemId, speak, Category.OTHER)
}
Category.SONGS.name -> {
- Song(itemId, loadingItemId, speak, Category.SONGS)
+ playSong(itemId, loadingItemId, speak, Category.SONGS)
}
Category.NAVIGATION.name -> {
- Navigation(itemId, loadingItemId, speak, Category.NAVIGATION)
+ navigate(itemId, loadingItemId, speak, Category.NAVIGATION)
}
Category.WEATHER.name -> {
- Weather(itemId, loadingItemId, speak, Category.WEATHER)
+ fetchWeather(itemId, loadingItemId, speak, Category.WEATHER)
}
Category.REMINDER.name -> {
- Reminder(itemId, loadingItemId, speak, Category.REMINDER)
+ setReminder(itemId, loadingItemId, speak, Category.REMINDER)
}
Category.ALARM.name -> {
- Alarm(itemId, loadingItemId, speak, Category.ALARM)
+ setAlarm(itemId, loadingItemId, speak, Category.ALARM)
}
}
}
- // Function to map a category name to a Category enum
- private fun getCategoryFromString(categoryName: String): Category {
- return Category.entries.find { it.name == categoryName.uppercase() } ?: Category.OTHER
- }
-
- private fun callAI(loadingItemId: Long, speak: Boolean, category: Category) {
- viewModelScope.launch {
- val response = withContext(Dispatchers.IO) {
- extractFromAI(sendQuestionToAPI_LM(MAIN_CONTEXT))
- }
-
- processResponse(response, loadingItemId, speak, category = category)
- }
- }
-
- // Suspending function for translation
- private suspend fun TranslatorManager.translateToEnglishSuspend(text: String): String? {
- return suspendCoroutine { continuation ->
- translateToEnglish(text) { translatedText ->
- continuation.resume(translatedText)
- }
- }
- }
-
- private fun cleanAndPunctuate(input: String): String {
- val trimmedInput = input.trim()
-
- // Return immediately if the input is empty
- if (trimmedInput.isEmpty()) return ""
-
- // Check if the last character is already a punctuation (either "." or "?")
- val lastChar = trimmedInput.last()
- return if (lastChar == '.' || lastChar == '?') {
- trimmedInput
- } else {
- // List of common question words
- val questionWords = listOf("wh", "how", "can", "do", "is", "are", "does", "did", "will", "could", "should", "would")
- if (questionWords.any { trimmedInput.lowercase().startsWith(it) }) {
- "$trimmedInput?"
- } else {
- trimmedInput
- }
- }
- }
-
- private suspend fun processResponse(
- response: String?,
+ internal fun callAI(
loadingItemId: Long,
speak: Boolean,
category: Category,
- contentURL: String ="",
- navigationURI: URI = URI("")
) {
- response?.let {
- withContext(Dispatchers.Main) {
-
- val plaintext = markdownToPlainText(it)
-
- if (getIsTranslationEnabled()) {
- translatorManager.translateFromEnglish(plaintext) { translatedText ->
- val finalText = translatedText ?: plaintext
- chatList.removeAt(chatList.indexOfFirst { it.id == loadingItemId })
- addConversationItem(plaintext, finalText, false, category, contentURL, navigationURI)
- if(speak)
- speakResponse(finalText)
+ viewModelScope.launch {
+ var fullResponse = ""
+ var hasStarted = false
+
+ val speakBuffer = StringBuilder()
+ var processedAnswerLength = 0
+ var isFirstSentence = true
+
+ val systemContext = if (_isHandsFreeModeActive.value && (_isVisionModeActive.value || _isScreenModeActive.value)) {
+ HANDS_FREE_MODE_IMAGE_CONTEXT
+ } else {
+ MAIN_CONTEXT
+ }
+ try {
+ processChatCommandUseCase.getAiChatResponseStream(
+ systemContext = systemContext,
+ chatHistory = chatList.toList(),
+ isHandsFreeActive = _isHandsFreeModeActive.value,
+ isVisionActive = _isVisionModeActive.value,
+ isScreenActive = _isScreenModeActive.value
+ )
+ .collect { chunk ->
+ if (!hasStarted) {
+ hasStarted = true
+ val index = chatList.indexOfFirst { it.id == loadingItemId }
+ if (index != -1) {
+ val item = chatList[index].copy(
+ text = "",
+ isMe = false,
+ isLoading = false,
+ isStreaming = true,
+ category = category.name
+ )
+ chatList.set(index, item)
+ }
+ }
+ fullResponse += chunk
+ val index = chatList.indexOfFirst { it.id == loadingItemId }
+ if (index != -1) {
+ val item = chatList[index].copy(
+ text = fullResponse,
+ isMe = false,
+ isLoading = false,
+ isStreaming = true,
+ category = category.name
+ )
+ chatList.set(index, item)
+ }
+
+ if (speak) {
+ val actualAnswerText = Conversation(text = fullResponse, isMe = false).getActualAnswer()
+ if (actualAnswerText.length > processedAnswerLength) {
+ val newTokens = actualAnswerText.substring(processedAnswerLength)
+ processedAnswerLength = actualAnswerText.length
+ speakBuffer.append(newTokens)
+
+ var searchIndex = 0
+ while (searchIndex < speakBuffer.length) {
+ val boundaryIndex = findSentenceBoundary(speakBuffer, searchIndex)
+ if (boundaryIndex != -1) {
+ val sentence = speakBuffer.substring(0, boundaryIndex + 1)
+ speakBuffer.delete(0, boundaryIndex + 1)
+ searchIndex = 0
+
+ val cleanText = com.app.assistant.util.MarkdownUtils.markdownToPlainText(sentence).trim()
+ if (cleanText.isNotEmpty()) {
+ val queueMode = if (isFirstSentence) 0 else 1
+ isFirstSentence = false
+ speakResponse(cleanText, queueMode)
+ }
+ } else {
+ break
+ }
+ }
+ }
+ }
+ }
+ } catch (e: Exception) {
+ Log.e("MainViewModel", "Streaming error", e)
+ fullResponse = "Error occurred while streaming response."
+ }
+
+ val index = chatList.indexOfFirst { it.id == loadingItemId }
+ if (index != -1) {
+ val item = chatList[index].copy(
+ text = fullResponse,
+ isMe = false,
+ isLoading = false,
+ isStreaming = false,
+ category = category.name
+ )
+ chatList.set(index, item)
+ } else {
+ addConversationItem(fullResponse, false, category)
+ }
+
+ if (chatList.size <= 2) {
+ loadGroup()
+ }
+
+ if (speak) {
+ val remainingText = speakBuffer.toString().trim()
+ if (remainingText.isNotEmpty()) {
+ val cleanText = com.app.assistant.util.MarkdownUtils.markdownToPlainText(remainingText).trim()
+ if (cleanText.isNotEmpty()) {
+ val queueMode = if (isFirstSentence) 0 else 1
+ speakResponse(cleanText, queueMode)
}
- } else {
- chatList.removeAt(chatList.indexOfFirst { it.id == loadingItemId })
- addConversationItem(response, "", false, category, contentURL, navigationURI)
- if(speak)
- speakResponse(plaintext)
}
}
}
}
- private fun addConversationItem(englishText: String, translatedText: String, isUser: Boolean, category: Category, contentURL: String ="", navigationURI: URI = URI("")) {
- val conversation = Conversation(englishText = englishText, translatedText = translatedText, isMe = isUser, category = category.name, contentURL = contentURL, navigationURI = navigationURI)
- chatList.add(conversation)
- if(chatList.size<=2)
- loadGroup()
+ private fun findSentenceBoundary(buffer: StringBuilder, startIndex: Int): Int {
+ val len = buffer.length
+ for (i in startIndex until len) {
+ val c = buffer[i]
+ if (c == '\n' || c == '\r') {
+ return i
+ }
+ if (c == '.' || c == '?' || c == '!') {
+ if (i + 1 < len) {
+ val nextChar = buffer[i + 1]
+ if (nextChar.isWhitespace()) {
+ if (c == '.') {
+ if (isAbbreviationOrDecimal(buffer, i)) {
+ continue
+ }
+ }
+ return i
+ }
+ }
+ }
+ }
+ return -1
}
- private fun speakResponse(
- plaintext: String
+ private fun isAbbreviationOrDecimal(buffer: StringBuilder, dotIndex: Int): Boolean {
+ var start = dotIndex - 1
+ while (start >= 0 && buffer[start].isLetter()) {
+ start--
+ }
+ val word = buffer.substring(start + 1, dotIndex).lowercase()
+ val abbreviations = setOf(
+ "mr", "mrs", "ms", "dr", "prof", "sr", "jr", "eg", "ie", "vs", "etc", "st", "co",
+ "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p",
+ "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"
+ )
+ return word in abbreviations
+ }
+
+ internal fun processResponse(
+ response: String?,
+ loadingItemId: Long,
+ speak: Boolean,
+ category: Category,
+ contentURL: String = "",
+ navigationURI: URI = URI(""),
) {
- textToSpeech.speak(plaintext, TextToSpeech.QUEUE_FLUSH, null, "left_for_now_id")
- textToSpeech.setOnUtteranceProgressListener(object : UtteranceProgressListener() {
-
- override fun onDone(utteranceId: String?) {
- _isSpeaking.value = false
+ response?.let {
+ val index = chatList.indexOfFirst { item -> item.id == loadingItemId }
+ if (index != -1) {
+ chatList.removeAt(index)
}
-
- @Deprecated("Deprecated in Java")
- override fun onError(utteranceId: String?) {
- _isSpeaking.value = false
+ addConversationItem(response, false, category, contentURL, navigationURI)
+ if (speak) {
+ val conversationTemp = Conversation(text = response, isMe = false)
+ val answerToSpeak = com.app.assistant.util.MarkdownUtils.markdownToPlainText(conversationTemp.getActualAnswer())
+ if (answerToSpeak.isNotBlank()) {
+ speakResponse(answerToSpeak)
+ }
}
+ }
+ }
- override fun onStart(utteranceId: String?) {
- _isSpeaking.value = true
- }
- })
+ internal fun addConversationItem(
+ text: String,
+ isUser: Boolean,
+ category: Category,
+ contentURL: String = "",
+ navigationURI: URI = URI(""),
+ ) {
+ val conversation = Conversation(
+ text = text,
+ isMe = isUser,
+ category = category.name,
+ contentURL = contentURL,
+ navigationURI = navigationURI,
+ )
+ chatList.add(conversation)
+ if (chatList.size <= 2) {
+ loadGroup()
+ }
+ }
+
+ internal fun speakResponse(plaintext: String, queueMode: Int = 0) {
+ viewModelScope.launch {
+ _uiEvent.send(UIEvent.SpeakText(plaintext, queueMode))
+ }
}
// Function to clear the conversation list
@@ -574,410 +850,97 @@ class MainViewModel(application: Application, private val speak: Boolean) : Andr
}
}
- //Function to start new chat
- fun newChat(){
+ // Function to start new chat
+ fun newChat() {
viewModelScope.launch {
chatList.clear()
repository.currentGroupId = -1L
}
}
- fun deleteMessage(index: Int){
- if(index!=null){
+ fun deleteGroup(groupId: Long) {
+ viewModelScope.launch {
+ _isDeletingChat.value = true
+ repository.deleteGroup(groupId)
+ if (repository.currentGroupId == groupId) {
+ chatList.clear()
+ repository.currentGroupId = -1L
+ }
+ loadGroup()
+ _isDeletingChat.value = false
+ }
+ }
+
+ fun deleteMessage(index: Int) {
+ if (index != null) {
chatList.removeAt(index)
}
}
- private suspend fun youtubeApiCall(query: String): Pair{
- val client = OkHttpClient()
- val deferredResult = CompletableDeferred>()
-
- val urlBuilder = "https://www.googleapis.com/youtube/v3/search".toHttpUrlOrNull()?.newBuilder()
- ?.addQueryParameter("key", loadYoutubeKey())
- ?.addQueryParameter("q", query)
- ?.addQueryParameter("type", "video")
- ?.addQueryParameter("part", "snippet")
- ?.addQueryParameter("maxResults","1")
- ?.build()
-
- val request = Request.Builder()
- .url(urlBuilder.toString())
- .build()
-
- client.newCall(request).enqueue(object : Callback {
- override fun onFailure(call: Call, e: IOException) {
- println("Request failed: $e")
- deferredResult.completeExceptionally(e)
- }
-
- override fun onResponse(call: Call, response: Response) {
- if (response.isSuccessful) {
- val body = response.body?.string()
- if(body != null)
- deferredResult.complete(extractVideoIdAndThumbnail(body))
- else
- deferredResult.complete(Pair("",""))
- } else {
- println("Request unsuccessful: ${response.code}")
- if(response.code == 403)
- deferredResult.complete(Pair("Missing API Key",""))
- else
- deferredResult.complete(Pair("",""))
- }
- response.close();
- }
- })
-
- return deferredResult.await()
+ fun startSpeechRecognition() {
+ viewModelScope.launch {
+ _uiEvent.send(UIEvent.StartSpeechRecognition)
+ }
}
- fun extractVideoIdAndThumbnail(jsonString: String): Pair {
- val jsonObject = JSONObject(jsonString)
- val itemsArray = jsonObject.optJSONArray("items") ?: return Pair("", "")
-
- if (itemsArray.length() > 0) {
- val firstItem = itemsArray.optJSONObject(0) ?: return Pair("", "")
- val idObject = firstItem.optJSONObject("id") ?: return Pair("", "")
-
- // Make sure it's a video, not a channel
- if (idObject.optString("kind") == "youtube#video") {
- val videoId = idObject.optString("videoId", "")
- val thumbnailUrl = firstItem.optJSONObject("snippet")
- ?.optJSONObject("thumbnails")
- ?.optJSONObject("high")
- ?.optString("url", "") ?: ""
-
- return Pair(videoId, thumbnailUrl)
- }
+ fun onSpeechRecognized(recognizedText: String) {
+ _question.value = recognizedText
+ if (recognizedText.isNotBlank()) {
+ processQuestion(speak = true)
}
-
- return Pair("", "") // Return empty if no valid video is found
}
- private suspend fun sendToAI(messagesArray: JSONArray): String {
- val client = OkHttpClient.Builder()
- .connectTimeout(60, TimeUnit.SECONDS)
- .readTimeout(60, TimeUnit.SECONDS)
- .writeTimeout(60, TimeUnit.SECONDS)
- .build()
-
- val requestBodyJson = JSONObject().apply {
- put("model", "meta-llama/llama-4-maverick-17b-128e-instruct")
- put("messages", messagesArray)
- put("temperature", 1)
- put("top_p", 1)
- put("stop", null as Any?)
- }
-
- val requestBody = requestBodyJson.toString()
- .toRequestBody("application/json".toMediaTypeOrNull())
-
- val request = Request.Builder()
- .url("https://api.groq.com/openai/v1/chat/completions")
- .addHeader("Authorization", "Bearer ${loadChatKey()}")
- .addHeader("Content-Type", "application/json")
- .post(requestBody)
- .build()
-
- return withContext(Dispatchers.IO) {
- try {
- client.newCall(request).execute().use { response ->
- if (!response.isSuccessful) {
- if (response.code == 401) {
- // API key is likely missing or invalid
- return@withContext """{"choices":[{"message":{"content":"Your chat API key is missing or invalid."}}]}"""
- }
- throw IOException("Unexpected code $response")
+ fun onSpeechRecognizedWithVision(recognizedText: String, speechStartTimestamp: Long) {
+ viewModelScope.launch(Dispatchers.IO) {
+ val optimalFrame = com.app.assistant.camera.VisualBufferManager.getOptimalFrame(speechStartTimestamp)
+ if (optimalFrame != null) {
+ val attachment = saveFrameAsAttachment(optimalFrame.jpegBytes)
+ withContext(Dispatchers.Main) {
+ if (attachment != null) {
+ _selectedAttachments.value = listOf(attachment)
}
- response.body!!.string()
- }
- } catch (e: UnknownHostException) {
- """{"choices":[{"message":{"content":"Seems this device is offline. Maybe try checking data connection."}}]}"""
- } catch (e: Exception) {
- e.printStackTrace()
- """{"choices":[{"message":{"content":"Seems some issue in my server. Please try again."}}]}"""
- }
- }
- }
-
- fun extractFromAI(response: String): String? {
- try {
- if (response.isNotEmpty()) {
- val jsonObject = JSONObject(response)
-
- val choicesArray = jsonObject.getJSONArray("choices")
- if (choicesArray.length() > 0) {
- val choiceObject = choicesArray.getJSONObject(0)
- val messageObject = choiceObject.getJSONObject("message")
- var content = messageObject.getString("content")
-
- // Regex to remove tags and the content inside
- val thinkTagRegex = Regex(".*? ", RegexOption.DOT_MATCHES_ALL)
- content = content.replace(thinkTagRegex, "") // Remove ...
-
- // Remove any standalone opening or closing tags
- content = content.replace("", "").replace(" ", "")
-
- return content.trim() // Clean and return the result
- } else {
- return ""
+ _question.value = recognizedText
+ processQuestion(speak = true)
}
} else {
- return "Seems some issue in my server. Please try again."
- }
- } catch (e: Exception) {
- e.printStackTrace()
- return null
- }
- }
-
-
- private suspend fun sendQuestionToAPI_LM(system_context: String): String{
- val messagesArray = JSONArray()
-
- messagesArray.put(JSONObject().put("role", "system").put("content", system_context))
-
- if(chatList.isNotEmpty()) {
- for (item in chatList) {
- val role = if (item.isMe) "user" else "assistant"
- val messageObject = JSONObject().put("role", role).put("content", item.englishText)
- messagesArray.put(messageObject)
- }
- }
- val response = sendToAI(messagesArray)
- return response
- }
-
- private fun startSpeechRecognition() {
- globalContext?.let { context ->
- startSpeechToText(
- context,
- onResult = { recognizedText ->
- question.value = recognizedText
- if (question.value.isNotEmpty()) {
- processQuestion(context = context, speak = true) // Process the question
- }
- },
- onPartialResult = { recognizedText ->
- question.value = recognizedText
+ withContext(Dispatchers.Main) {
+ _question.value = recognizedText
+ processQuestion(speak = true)
}
+ }
+ }
+ }
+
+ private fun saveFrameAsAttachment(jpegBytes: ByteArray): com.app.assistant.model.Attachment? {
+ return try {
+ val context = getApplication().applicationContext
+ val fileName = "vision_frame_${System.currentTimeMillis()}.jpg"
+ val mimeType = "image/jpeg"
+ val iv = com.app.assistant.db.EncryptionUtil.generateIV()
+ val encryptedBytes = com.app.assistant.db.EncryptionUtil.encryptFile(jpegBytes, iv)
+
+ val attachmentsDir = java.io.File(context.filesDir, "attachments")
+ if (!attachmentsDir.exists()) {
+ attachmentsDir.mkdirs()
+ }
+
+ val targetFile = java.io.File(attachmentsDir, "file_${System.currentTimeMillis()}_${java.util.UUID.randomUUID()}.enc")
+ targetFile.writeBytes(encryptedBytes)
+
+ com.app.assistant.model.Attachment(
+ filePath = targetFile.absolutePath,
+ mimeType = mimeType,
+ fileName = fileName,
+ iv = iv
)
+ } catch (e: Exception) {
+ Log.e("MainViewModel", "Failed to save vision frame as attachment", e)
+ null
}
}
- fun startSpeechToText(context: Context, onResult: (String) -> Unit, onPartialResult: (String) -> Unit) {
- if (ContextCompat.checkSelfPermission(context, RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
- // Request the RECORD_AUDIO permission.
- ActivityCompat.requestPermissions(context as Activity, arrayOf(RECORD_AUDIO), 101)
- return
- }
-
- // Ensure Bluetooth SCO is started for better compatibility with Bluetooth headsets
- val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
- audioManager.mode = AudioManager.MODE_IN_CALL
- audioManager.isBluetoothScoOn = true
- audioManager.startBluetoothSco()
-
- val originalRingerMode = audioManager.ringerMode // Save the current ringer mode
-
-
- if(textToSpeech.isSpeaking)
- textToSpeech.stop()
-
- val speechRecognizer = SpeechRecognizer.createSpeechRecognizer(context)
- val speechRecognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
- speechRecognizerIntent.putExtra(
- RecognizerIntent.EXTRA_LANGUAGE_MODEL,
- RecognizerIntent.LANGUAGE_MODEL_FREE_FORM
- )
- speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true);
- if (getIsTranslationEnabled())
- speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE,getLocaleCode(getActiveLanguageCode()))
- else
- speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault())
-
- speechRecognizer.setRecognitionListener(object : RecognitionListener {
- override fun onReadyForSpeech(bundle: Bundle?) {
- // Put the phone in vibrate mode temporarily to suppress default beep
- if(originalRingerMode == AudioManager.RINGER_MODE_NORMAL)
- audioManager.ringerMode = AudioManager.RINGER_MODE_VIBRATE
- }
- override fun onBeginningOfSpeech() {
- _isListening.value = true
- }
- override fun onRmsChanged(v: Float) {}
- override fun onBufferReceived(bytes: ByteArray?) {}
- override fun onEndOfSpeech() {
- _isListening.value = false
- if(originalRingerMode == AudioManager.RINGER_MODE_NORMAL) {
- CoroutineScope(Dispatchers.Default).launch {
- delay(800)
- audioManager.ringerMode = originalRingerMode
- }
- }
- }
-
- override fun onError(errorCode: Int) {
- val message = when (errorCode) {
- SpeechRecognizer.ERROR_AUDIO -> "Audio recording error"
- SpeechRecognizer.ERROR_CLIENT -> "Client side error"
- SpeechRecognizer.ERROR_INSUFFICIENT_PERMISSIONS -> "Insufficient permissions"
- SpeechRecognizer.ERROR_NETWORK -> "Network error"
- SpeechRecognizer.ERROR_NO_MATCH -> "No match found"
- SpeechRecognizer.ERROR_RECOGNIZER_BUSY -> "Recognition service is busy"
- SpeechRecognizer.ERROR_SERVER -> "Server error"
- SpeechRecognizer.ERROR_SPEECH_TIMEOUT -> "Speech input timeout"
- else -> "Unknown error"
- }
- //Toast.makeText(context, "Error occurred: $message", Toast.LENGTH_SHORT).show()
- _isListening.value = false
- if(originalRingerMode == AudioManager.RINGER_MODE_NORMAL) {
- CoroutineScope(Dispatchers.Default).launch {
- delay(800)
- audioManager.ringerMode = originalRingerMode
- }
- }
- }
-
- override fun onResults(bundle: Bundle?) {
- _isListening.value = false
- val recognizedText = bundle?.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)?.get(0) ?: ""
- onResult(recognizedText)
- }
-
- override fun onPartialResults(bundle: Bundle) {
- val recognizedText = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)?.get(0) ?: ""
- onPartialResult(recognizedText)
- }
- override fun onEvent(i: Int, bundle: Bundle?) {}
-
- })
- speechRecognizer.startListening(speechRecognizerIntent)
-
- // Clean up resources when done
- context.registerComponentCallbacks(object : ComponentCallbacks2 {
- override fun onTrimMemory(level: Int) {
- if (level == ComponentCallbacks2.TRIM_MEMORY_UI_HIDDEN) {
- // Stop Bluetooth SCO
- audioManager.stopBluetoothSco()
- audioManager.isBluetoothScoOn = false
- speechRecognizer.destroy()
- }
- }
-
- override fun onConfigurationChanged(newConfig: Configuration) {}
- override fun onLowMemory() {}
- })
- }
-
- fun playCustomToneWithMutedBeep(context: Context, customToneResId: Int) {
- val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
- val originalRingerMode = audioManager.ringerMode // Save the current ringer mode
-
- if(originalRingerMode == AudioManager.RINGER_MODE_NORMAL) {
-
- // Put the phone in vibrate mode temporarily to suppress default beep
- audioManager.ringerMode = AudioManager.RINGER_MODE_VIBRATE
-
- // Play custom tone
- val mediaPlayer = MediaPlayer.create(context, customToneResId)
- mediaPlayer.setOnCompletionListener {
- // Restore the original ringer mode after the tone finishes playing
- audioManager.ringerMode = originalRingerMode
- mediaPlayer.release()
- }
- mediaPlayer.start()
- }
- }
-
- @SuppressLint("ServiceCast")
- fun MuteBeep(context: Context) {
- val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
- //val vibrator = getSystemService(context, Vibrator::class.java)
-
- val originalRingerMode = audioManager.ringerMode // Save the current ringer mode
-
- // Put the phone in vibrate mode temporarily to suppress default beep
- audioManager.ringerMode = AudioManager.RINGER_MODE_VIBRATE
-
- // Vibrate for minimal duration
-// vibrator?.let {
-// if (Build.VERSION.SDK_INT >= 26) {
-// it.vibrate(VibrationEffect.createOneShot(100, VibrationEffect.DEFAULT_AMPLITUDE))
-// } else {
-// @Suppress("DEPRECATION")
-// it.vibrate(100)
-// }
-// }
-
- // Restore the original ringer mode after a short delay
- Handler(Looper.getMainLooper()).postDelayed({
- audioManager.ringerMode = originalRingerMode
- }, 800) // Wait for a short time to ensure vibration is completed
- }
-
-
- fun getLocaleCode(languageCode: String): String {
- val languageCountryMapping = mapOf(
- "af" to "ZA", "sq" to "AL", "ar" to "SA", "be" to "BY", "bg" to "BG",
- "bn" to "BD", "ca" to "ES", "zh" to "CN", "hr" to "HR", "cs" to "CZ",
- "da" to "DK", "nl" to "NL", "en" to "US", "eo" to "EO", "et" to "EE",
- "fi" to "FI", "fr" to "FR", "gl" to "ES", "ka" to "GE", "de" to "DE",
- "el" to "GR", "gu" to "IN", "ht" to "HT", "he" to "IL", "hi" to "IN",
- "hu" to "HU", "is" to "IS", "id" to "ID", "ga" to "IE", "it" to "IT",
- "ja" to "JP", "kn" to "IN", "ko" to "KR", "lt" to "LT", "lv" to "LV",
- "mk" to "MK", "mr" to "IN", "ms" to "MY", "mt" to "MT", "no" to "NO",
- "fa" to "IR", "pl" to "PL", "pt" to "BR", "ro" to "RO", "ru" to "RU",
- "sk" to "SK", "sl" to "SI", "es" to "ES", "sv" to "SE", "sw" to "KE",
- "tl" to "PH", "ta" to "IN", "te" to "IN", "th" to "TH", "tr" to "TR",
- "uk" to "UA", "ur" to "PK", "vi" to "VN", "cy" to "GB"
- )
-
- // Retrieve the country code based on the language or fall back to Locale's default country
- val countryCode = languageCountryMapping[languageCode] ?: Locale.getDefault().country
- return "$languageCode-$countryCode"
- }
-
- fun markdownToPlainText(markdown: String): String {
- var plainText = markdown
-
- // Basic Markdown stripping (simplified)
- plainText = plainText.replace(Regex("#+ "), "") // Remove headers
- plainText = plainText.replace(Regex("\\*\\*(.*?)\\*\\*"), "$1") // Bold
- plainText = plainText.replace(Regex("\\*(.*?)\\*"), "$1") // Italic
- plainText = plainText.replace(Regex("!\\[.*?\\]\\(.*?\\)"), "") // Images
- plainText = plainText.replace(Regex("\\[.*?\\]\\(.*?\\)"), "") // Links
- plainText = plainText.replace(Regex("`{1,3}.*?`{1,3}"), "") // Inline code
- plainText = plainText.replace(Regex("\\n\\n"), "\n") // Simplify newlines
-
- return plainText.trim()
- }
-
- override fun onCleared() {
- super.onCleared()
- shutdownResources()
- }
-
- // Update the selected item
- fun onItemSelected(selectedLanguageCode: String) {
- _isLanguageLoading.value = true
- setupTranslator(selectedLanguageCode, true)
- triggerToast("Downloading translation model.");
- }
-
- private fun setupTranslator(selectedLanguageCode: String, showCompletionToast: Boolean = false) {
- translatorManager.setupTranslators(selectedLanguageCode) { success ->
- if (success) {
- updateActiveLanguageCode(selectedLanguageCode)
- _isLanguageLoading.value = false
- showBottomSheet.value = false
- if(showCompletionToast)
- triggerToast("Download completed, its recommended to use selected language keyboard.");
- } else {
- _isLanguageLoading.value = false
- triggerToast("Something went wrong, model download failed.")
- }
- }
+ fun onSpeechPartialResult(recognizedText: String) {
+ _question.value = recognizedText
}
private fun triggerToast(message: String) {
@@ -986,1095 +949,14 @@ class MainViewModel(application: Application, private val speak: Boolean) : Andr
}
}
- private fun isNegativeOrNotRequired(phrase: String): Boolean {
- val negativePatterns = listOf(
- Regex("\\bno\\b", RegexOption.IGNORE_CASE),
- Regex("\\bnah\\b", RegexOption.IGNORE_CASE),
- Regex("\\bnot\\b.*", RegexOption.IGNORE_CASE), // Matches "Not now", "Not anymore", etc.
- Regex("\\bnever\\b.*", RegexOption.IGNORE_CASE), // Matches "Never mind", "Never again", etc.
- Regex("\\bforget\\b.*", RegexOption.IGNORE_CASE), // Matches "Forget it", "Forget about it"
- Regex("\\bleave\\b.*", RegexOption.IGNORE_CASE), // Matches "Leave it"
- Regex("\\bdrop\\b.*", RegexOption.IGNORE_CASE), // Matches "Drop it"
- Regex("\\bcancel\\b", RegexOption.IGNORE_CASE),
- Regex("\\babort\\b", RegexOption.IGNORE_CASE),
- Regex("\\bstop\\b", RegexOption.IGNORE_CASE),
- Regex("\\bquit\\b", RegexOption.IGNORE_CASE),
- Regex("\\bdisregard\\b", RegexOption.IGNORE_CASE),
- Regex("changed my mind", RegexOption.IGNORE_CASE),
- Regex("don't bother", RegexOption.IGNORE_CASE),
- Regex("let it go", RegexOption.IGNORE_CASE),
- Regex("scratch that", RegexOption.IGNORE_CASE),
- Regex("just kidding", RegexOption.IGNORE_CASE),
- )
-
- return negativePatterns.any { it.containsMatchIn(phrase) }
- }
-
- // Function to clear data in lock states
- private fun clearLockStateData() {
- // Reset specific fields for all lock states
+ internal fun clearLockStateData() {
if (lockState is LockState.LockAlarm) {
(lockState as LockState.LockAlarm).day = null
}
- // Add similar reset logic for other lock states if needed in the future
}
- //For call
- private fun Call(itemId: Long, loadingItemId: Long, speak: Boolean, category: Category){
- if(chatList.isNotEmpty()) {
- val prompt = chatList.find { it.id == itemId }?.englishText?.replace("\\p{Punct}+".toRegex(), "")?:return
- val permissions = arrayOf(READ_CONTACTS, CALL_PHONE)
- val requiredPermissions = permissions.filter {
- ContextCompat.checkSelfPermission(globalContext!!, it) != PackageManager.PERMISSION_GRANTED
- }
-
- if (requiredPermissions.isNotEmpty()) {
- ActivityCompat.requestPermissions(
- globalContext as Activity,
- requiredPermissions.toTypedArray(),102
- )
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.permissionContactsCall), loadingItemId, speak, Category.OTHER)
- }
- return
- }
-
- viewModelScope.launch {
- val (success, result) = searchContactAndMakeCall(prompt)
-
- if (success) {
- val contact = result as Contact
- processResponse(contact.name, loadingItemId, false, category, navigationURI = URI("tel:${contact.phoneNumber.replace(" ","")}"))
- } else {
- processResponse(result.toString(), loadingItemId, speak, Category.OTHER)
- }
- }
- }
+ override fun onCleared() {
+ super.onCleared()
+ shutdownResources()
}
-
- @SuppressLint("Range")
- private suspend fun searchContactAndMakeCall(prompt: String): Pair {
- val contentResolver = getApplication().contentResolver
-
- val contactList = mutableListOf()
-
- withContext(Dispatchers.IO) {
- val cursor = contentResolver.query(
- ContactsContract.Data.CONTENT_URI,
- arrayOf(
- ContactsContract.Data.CONTACT_ID,
- ContactsContract.Data.DISPLAY_NAME,
- ContactsContract.CommonDataKinds.Phone.NUMBER
- ),
- "${ContactsContract.Data.MIMETYPE} =? AND ${ContactsContract.Data.CONTACT_ID} IS NOT NULL",
- arrayOf(ContactsContract.CommonDataKinds.Phone.CONTENT_ITEM_TYPE),
- null
- )
-
- cursor?.use { contactCursor ->
- while (contactCursor.moveToNext()) {
- val contactId = contactCursor.getString(contactCursor.getColumnIndex(ContactsContract.Data.CONTACT_ID))
- val name = contactCursor.getString(contactCursor.getColumnIndex(ContactsContract.Data.DISPLAY_NAME))
- val phoneNumber = contactCursor.getString(contactCursor.getColumnIndex(ContactsContract.CommonDataKinds.Phone.NUMBER))
- contactList.add(Contact(contactId, name, phoneNumber))
- }
- }
- }
-
- val contactName = cleanAndExtractName(prompt)
-
- val bestMatch = findBestMatch(contactName, contactList)
-
- if (bestMatch != null) {
- try {
- val callIntent = Intent(Intent.ACTION_CALL)
- callIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
- callIntent.data = Uri.parse("tel:${bestMatch.phoneNumber}")
- globalContext?.startActivity(callIntent)
- return Pair(true, bestMatch)
- } catch (e: Exception) {
- Log.e("MainViewModel", "Error making call:", e)
- return Pair(false, "Sorry, failed to make call. Please try again.")
- }
- } else {
- return Pair(false, "I cannot find such contact, please try again.")
- }
- }
-
- private fun cleanAndExtractName(command: String): String {
- val stopWords = setOf("can", "you", "make", "call", "dial", "number", "to", "please", "the", "a", "on")
- val words = command.lowercase().trim().split(" ")
- return words.filter { !stopWords.contains(it) }.joinToString(" ")
- }
-
- private fun findBestMatch(inputString: String, nameList: List, similarityThreshold: Double = 0.7): Contact? {
- if(inputString.isEmpty()){
- return null
- }
- val inputLower = inputString.lowercase().trim()
- var exactMatch: Contact? = null
- var bestSubstringMatch: Contact? = null
- var bestFuzzyMatch: Contact? = null
- var bestFuzzyScore = 0.0
-
- for (contact in nameList) {
- val nameLower = contact.name.lowercase().trim()
-
- // Check for exact match
- if (inputLower == nameLower) {
- // Return immediately if there's an exact match
- return contact
- }
-
- // Check for exact match within tokens
- val tokens = nameLower.split(" ")
- for (token in tokens) {
- when {
- inputLower == token -> {
- exactMatch = contact // Keep track of the exact match
- }
- token.contains(inputLower) -> {
- bestSubstringMatch = contact // Keep track of the substring match if no exact match is found
- }
- else -> {
- // Perform fuzzy matching as a last resort
- val similarityScore = stringSimilarity(inputLower, token)
- if (similarityScore > bestFuzzyScore) {
- bestFuzzyScore = similarityScore
- bestFuzzyMatch = contact
- }
- }
- }
- }
- }
-
- // Return in order of priority: exact match, substring match, fuzzy match
- return when {
- exactMatch != null -> exactMatch
- bestSubstringMatch != null -> bestSubstringMatch
- bestFuzzyScore >= similarityThreshold -> bestFuzzyMatch
- else -> null // No match found if similarity score is too low
- }
- }
-
- private fun stringSimilarity(a: String, b: String): Double {
- val matcher = a.commonPrefixWith(b).length.toDouble() / max(a.length, b.length).toDouble()
- return matcher
- }
- //End Call
-
- //For Song
- private fun Song(itemId: Long, loadingItemId: Long, speak: Boolean, category: Category) {
- try {
- val prompt = chatList.find { it.id == itemId }?.englishText?.replace("\\p{Punct}+".toRegex(), "")?: return
- if (!prompt.lowercase().contains("play ")) {
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.songNotFound),loadingItemId, speak,
- Category.OTHER)
- }
- return
- }
- val searchQuery = prompt.lowercase().substringAfter("play").trim()
- viewModelScope.launch {
- val (videoId, thumbnailUrl) = youtubeApiCall(searchQuery)
- if (videoId.isEmpty()) {
- processResponse(getRandomResponse(ResponseStrings.songNotFound), loadingItemId, speak, Category.OTHER)
- } else if(videoId == "Missing API Key"){
- val intent = Intent(Intent.ACTION_SEARCH)
- intent.setPackage("com.google.android.youtube")
- intent.putExtra("query", searchQuery)
- intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
- globalContext?.startActivity(intent)
- processResponse("Your Youtube API key is missing or invalid.", loadingItemId, speak, Category.OTHER)
- } else {
- val intent = Intent(Intent.ACTION_VIEW, "http://www.youtube.com/watch?v=$videoId".toUri())
- intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
- globalContext?.startActivity(intent)
- processResponse("Playing $searchQuery", loadingItemId, speak, category, thumbnailUrl, URI("http://www.youtube.com/watch?v=$videoId"))
- }
- }
- } catch (e: Exception) {
- Log.d("Exception", e.message.toString())
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.genericError),loadingItemId, speak, Category.OTHER)
- }
- }
- }
- //End Song
-
- //For Navigation
- private fun Navigation(itemId: Long, loadingItemId: Long, speak: Boolean, category: Category){
- try {
- if (chatList.isNotEmpty()) {
- val prompt = chatList.find { it.id == itemId }?.englishText?.replace("\\p{Punct}+".toRegex(), "")?:return
- val location = extractLocation(prompt)
- viewModelScope.launch {
- if (location!=null) {
- val gmmIntentUri =
- Uri.parse("google.navigation:q=$location")
- val mapIntent = Intent(Intent.ACTION_VIEW, gmmIntentUri)
- mapIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
- mapIntent.setPackage("com.google.android.apps.maps")
- globalContext?.startActivity(mapIntent)
- val encodedURI = URLEncoder.encode(location, StandardCharsets.UTF_8.toString())
- processResponse("Navigating to $location.", loadingItemId, speak, category, navigationURI = URI("google.navigation:q=$encodedURI"))
- }else{
- processResponse(getRandomResponse(ResponseStrings.locationNotFound), loadingItemId, speak, Category.OTHER)
- }
- }
- }else{
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.locationNotFound), loadingItemId, speak, Category.OTHER)
- }
- }
- }catch (e: Exception){
- Log.d("Exception", e.message.toString())
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.genericError), loadingItemId, speak, Category.OTHER)
- }
- }
- }
-
- private fun extractLocation(command: String): String? {
- // Regex pattern to capture location phrases based on patterns observed in the examples
- val regex = Regex(
- "(?<=to |find |show me |give me directions to |navigate me to |how do I get to |I'm on my way to |start a navigation to |where is |help me find |traffic like on the way to |way to )" +
- "([A-Za-z0-9\\s&]+)"
- )
-
- // Attempt to find the location in the command string
- val matchResult = regex.find(command)
- return matchResult?.value?.trim()
- }
- //End Navigation
-
- //For Weather
- private fun Weather(itemId: Long, loadingItemId: Long, speak: Boolean, category: Category){
- try {
- if(chatList.isNotEmpty()){
- val prompt = chatList.find { it.id == itemId }?.englishText?.replace("\\p{Punct}+".toRegex(), "")?: return
- val location = extractPlaceName(prompt)
- if(location.isNotEmpty()){
- viewModelScope.launch {
- val coordinates = getCoordinatesFromCity(location)
- if(coordinates!=null) {
- val (lat, long) = coordinates
- val weatherData = getWeatherData(lat, long)
- if(weatherData!=null){
- weatherData.remove("latitude");
- weatherData.remove("longitude");
- val response = withContext(Dispatchers.IO) {
- extractFromAI(sendQuestionToAI(prompt, weatherData.toString()))
- }
- if(!response.isNullOrEmpty()){
- processResponse(response, loadingItemId, speak, category, navigationURI = URI("https://www.google.com/search?q=weather+${location}"))
- }
- }else{
- processResponse(getRandomResponse(ResponseStrings.weatherReportUnavailable), loadingItemId, speak, Category.OTHER)
- }
- }else{
- processResponse(getRandomResponse(ResponseStrings.locationNotFound), loadingItemId, speak, Category.OTHER)
- }
- }
- }else{
- if (globalContext?.let { ContextCompat.checkSelfPermission(it, Manifest.permission.ACCESS_COARSE_LOCATION) } != PackageManager.PERMISSION_GRANTED) {
- // Request permission if it hasn't been granted
- ActivityCompat.requestPermissions(
- globalContext as Activity,
- arrayOf(Manifest.permission.ACCESS_COARSE_LOCATION),
- 103
- )
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.permissionLocation), loadingItemId, speak, Category.OTHER)
- }
- }else{
- val locationManager = globalContext?.getSystemService(Context.LOCATION_SERVICE) as LocationManager
- val gpsEnabled = locationManager.isProviderEnabled(LocationManager.GPS_PROVIDER)
-
- if (!gpsEnabled) {
- promptEnableLocation()
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.locationServiceOff), loadingItemId, speak, Category.OTHER)
- }
- } else {
- // GPS is on and permission is granted
- val fusedLocationClient = LocationServices.getFusedLocationProviderClient(globalContext!!)
- // Create a LocationRequest
- val locationRequest = LocationRequest.Builder(Priority.PRIORITY_HIGH_ACCURACY, 1000).setWaitForAccurateLocation(true).setMaxUpdates(1).build()
- // Create a LocationCallback
- val locationCallback = object : LocationCallback() {
- override fun onLocationResult(locationResult: LocationResult) {
- val location = locationResult.lastLocation
- if (location != null) {
- val lat = location.latitude
- val long = location.longitude
- val city = getCityNameFromLocation(lat, long)
- // Process the location
- viewModelScope.launch {
- val weatherData = getWeatherData(lat, long)
- if(weatherData!=null){
- weatherData.remove("latitude");
- weatherData.remove("longitude");
- val response = withContext(Dispatchers.IO) {
- extractFromAI(sendQuestionToAI(prompt, weatherData.toString()))
- }
- if(!response.isNullOrEmpty()){
- processResponse(response, loadingItemId, speak, Category.WEATHER, navigationURI = URI("https://www.google.com/search?q=weather+${URLEncoder.encode(city, StandardCharsets.UTF_8.toString())}"))
- }
- }else{
- processResponse(getRandomResponse(ResponseStrings.weatherReportUnavailable), loadingItemId, speak, Category.OTHER)
- }
- }
- } else {
- // If location is null, handle the failure
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.weatherReportUnavailable),loadingItemId, speak,
- Category.OTHER)
- }
- }
- // Remove location updates after receiving a result
- fusedLocationClient.removeLocationUpdates(this)
- }
-
- override fun onLocationAvailability(locationAvailability: LocationAvailability) {
- if (!locationAvailability.isLocationAvailable) {
- // Handle the case where location is unavailable
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.locationUnknownSuggestCity),loadingItemId, speak,
- Category.OTHER)
- }
- }
- }
- }
- // Request a single location update
- fusedLocationClient.requestLocationUpdates(locationRequest, locationCallback, Looper.getMainLooper())
- }
- }
- }
- }
- }catch (e: Exception){
- Log.d("Exception", e.message.toString())
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.genericError), loadingItemId, speak, Category.OTHER)
- }
- }
- }
-
- private fun extractPlaceName(inputText: String): String {
- // Pattern to match place names after "in" or "for", excluding trailing words like "today", "tomorrow", etc.
- val pattern = Regex("\\b(?:in|for)\\s+([A-Za-z]+(?:\\s+[A-Za-z]+)*)(?=\\b(?:\\s+(today|now|tomorrow|report|forecast|currently|evening|morning|afternoon)\\b|\\s*$))")
-
- // Search for a match in the input text
- val match = pattern.find(inputText)
-
- // Check if the captured group does not match excluded words (e.g., "tomorrow") and return it
- val placeName = match?.groups?.get(1)?.value?.trim()
- return if (placeName != null && !placeName.lowercase().matches(Regex("today|now|tomorrow|report|forecast|currently|evening|morning|afternoon"))) {
- placeName
- } else {
- ""
- }
- }
-
- private suspend fun getCoordinatesFromCity(cityName: String): Pair? {
- val client = OkHttpClient()
- val deferredResult = CompletableDeferred?>()
-
- val urlBuilder = "https://geocoding-api.open-meteo.com/v1/search".toHttpUrlOrNull()?.newBuilder()
- ?.addQueryParameter("name", cityName)
- ?.build()
-
- val request = Request.Builder()
- .url(urlBuilder.toString())
- .build()
-
- client.newCall(request).enqueue(object : Callback {
- override fun onFailure(call: Call, e: IOException) {
- println("Request failed: $e")
- deferredResult.completeExceptionally(e)
- }
-
- override fun onResponse(call: Call, response: Response) {
- if (response.isSuccessful) {
- val body = response.body?.string()?: return
- deferredResult.complete(extractCoordinates(body, cityName))
- } else {
- println("Request unsuccessful: ${response.code}")
- deferredResult.cancel()
- }
- }
- })
-
- return deferredResult.await()
- }
-
- private fun extractCoordinates(body: String, cityName: String): Pair? {
- val jsonObject = JSONObject(body)
- var latitude: Double? = null
- var longitude: Double? = null
-
- try {
-
- val results = jsonObject.getJSONArray("results")
-
- for (i in 0 until results.length()) {
- val result = results.getJSONObject(i)
- if (result.getString("name").equals(cityName, ignoreCase = true)) {
- latitude = result.getDouble("latitude")
- longitude = result.getDouble("longitude")
- break
- }
- }
-
- if (latitude == null || longitude == null && results.length() > 0) {
- val firstResult = results.getJSONObject(0)
- latitude = firstResult.getDouble("latitude")
- longitude = firstResult.getDouble("longitude")
- }
- }catch (e: Exception){
- e.message?.let { Log.d("extractCoordinates Excepetion", it) }
- }
-
- return if (latitude!= null && longitude!= null) Pair(latitude, longitude) else null
- }
-
- private fun getCityNameFromLocation(latitude: Double, longitude: Double): String? {
- val geocoder = globalContext?.let { Geocoder(it, Locale.getDefault()) }
- return try {
- val addresses = geocoder?.getFromLocation(latitude, longitude, 1)
- if (addresses?.isNotEmpty() == true) {
- addresses[0].locality // This gives the city name
- } else {
- null
- }
- } catch (e: Exception) {
- e.printStackTrace()
- null
- }
- }
-
- private suspend fun getWeatherData(latitude: Double, longitude: Double): JSONObject? {
- val client = OkHttpClient()
- val deferredResult = CompletableDeferred()
-
- val urlBuilder = "https://api.open-meteo.com/v1/forecast".toHttpUrlOrNull()?.newBuilder()
- ?.addQueryParameter("latitude", latitude.toString())
- ?.addQueryParameter("longitude", longitude.toString())
- ?.addQueryParameter("daily", "temperature_2m_max,temperature_2m_min,precipitation_sum,windspeed_10m_max")
- ?.addQueryParameter("hourly", "temperature_2m,precipitation,windspeed_10m")
- ?.addQueryParameter("timezone", "auto")
- ?.addQueryParameter("forecast_days", "10")
- ?.build()
-
- val request = Request.Builder()
- .url(urlBuilder.toString())
- .build()
-
- client.newCall(request).enqueue(object : Callback {
- override fun onFailure(call: Call, e: IOException) {
- println("Weather data request failed: $e")
- deferredResult.complete(null)
- }
-
- override fun onResponse(call: Call, response: Response) {
- if (response.isSuccessful) {
- val body = response.body?.string()?: return
- try {
- deferredResult.complete(JSONObject(body))
- } catch (e: JSONException) {
- println("Error parsing JSON: $e")
- deferredResult.complete(null)
- }
- } else {
- println("Weather data request unsuccessful: ${response.code}")
- deferredResult.complete(null)
- }
- }
- })
-
- return deferredResult.await()
- }
-
- private fun promptEnableLocation() {
- val locationRequest = LocationRequest.Builder(100, 10000) // 10-second interval
- .setMinUpdateIntervalMillis(5000) // Minimum interval between updates
- .build()
-
- val builder = LocationSettingsRequest.Builder().addLocationRequest(locationRequest)
- val settingsClient: SettingsClient = LocationServices.getSettingsClient(globalContext!!)
- val task: Task = settingsClient.checkLocationSettings(builder.build())
-
- task.addOnFailureListener { exception ->
- if (exception is ResolvableApiException) {
- // Show the native dialog to turn on GPS
- try {
- exception.startResolutionForResult(globalContext as Activity, 104)
- } catch (sendEx: IntentSender.SendIntentException) {
- Log.d("Location", "Error showing location settings dialog: ${sendEx.message}")
- }
- }
- }
- }
-
- private suspend fun sendQuestionToAI(question: String, weatherData:String): String{
- val messagesArray = JSONArray()
-
- val date = SimpleDateFormat("yyyy-MM-dd", Locale. getDefault(Locale. Category. FORMAT)).format(Calendar.getInstance().time)
- val time = SimpleDateFormat("HH:mm:ss", Locale. getDefault(Locale. Category. FORMAT)).format(Calendar.getInstance().time)
- val systemContext = "You are a smart weather assistant, up-to-date with the current date and time, which is $date at $time. Please respond with an answer to the user's question based on the latest weather data provided. No need to mention data or time; just answer naturally in 2 to 3 lines."
- messagesArray.put(JSONObject().put("role", "system").put("content", systemContext))
- messagesArray.put(JSONObject().put("role", "user").put("content", question + System.lineSeparator() + weatherData))
-
- val response = sendToAI(messagesArray)
- return response
- }
- //End Weather
-
- //For Alarm
- private fun Alarm(itemId: Long, loadingItemId: Long, speak: Boolean, category: Category){
- try {
- if (chatList.isNotEmpty()) {
- val prompt = chatList.find { it.id == itemId }?.englishText?.replace("[.?!]+".toRegex(), "")?:return
-
- val dayRegex = "(?i)\\b(today|tomorrow|next week|next weekend|((Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday)( morning| evening| night| afternoon)?))\\b".toRegex(RegexOption.IGNORE_CASE)
-
- val timeRegex = "(?i)\\b(\\d{1,2})(:(\\d{2}))?\\s*(AM|PM)?\\b".toRegex(RegexOption.IGNORE_CASE)
-
- val relativeTimeRegex = "(?i)\\b(\\d+)?\\s*(?:and\\s*(half))?\\s*(hours?|hrs?|mins?|minutes?|seconds?|secs?)\\s*(?:and\\s*(\\d+))?\\s*(mins?|minutes?|secs?|seconds?)?\\s*(from now|later|next)?\\b".toRegex(RegexOption.IGNORE_CASE)
-
-
- // Extract day, time, and relative time from the prompt
- val dayMatch = dayRegex.find(prompt)?.value?.lowercase(Locale.ROOT)
- val timeMatch = timeRegex.find(prompt)
- val relativeTimeMatch = relativeTimeRegex.find(prompt)
-
- if(timeMatch == null && relativeTimeMatch == null){
- viewModelScope.launch {
- lockState = LockState.LockAlarm(day = dayMatch)
- processResponse(getRandomResponse(ResponseStrings.promptForTime), loadingItemId, speak, Category.OTHER)
- }
- }else{
- viewModelScope.launch {
- setAlarmFromPrompt(dayMatch, timeMatch, relativeTimeMatch)
- processResponse(getRandomResponse(ResponseStrings.alarmSetSuccess), loadingItemId, speak, category)
- }
- }
- }
- }catch (e: Exception){
- Log.d("Exception", e.message.toString())
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.genericError), loadingItemId, speak, Category.OTHER)
- }
- }
- }
-
- private fun handleAlarmLockState (itemId: Long, loadingItemId: Long, speak: Boolean, state: LockState.LockAlarm){
- try{
- val prompt = chatList.find { it.id == itemId }?.englishText?.replace("[.?!]+".toRegex(), "")?:return
- val timeRegex = "(\\d{1,2})(:(\\d{2}))?\\s*(AM|PM)?".toRegex(RegexOption.IGNORE_CASE)
- val relativeTimeRegex = "(\\d+)\\s*(hours?|mins?|minutes?)\\s*(from now|later|next)?".toRegex(RegexOption.IGNORE_CASE)
-
- val dayMatch = state.day
- val timeMatch = timeRegex.find(prompt)
- val relativeTimeMatch = relativeTimeRegex.find(prompt)
-
- if(timeMatch == null && relativeTimeMatch == null){
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.invalidTime), loadingItemId, speak, Category.OTHER)
- }
- return
- }else{
- setAlarmFromPrompt(dayMatch, timeMatch, relativeTimeMatch)
- lockState = LockState.None
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.alarmSetSuccess), loadingItemId, speak, Category.ALARM)
- }
- }
- }catch (e: Exception){
- Log.d("Exception", e.message.toString())
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.genericError), loadingItemId, speak, Category.OTHER)
- }
- }
- }
-
- private fun setAlarmFromPrompt(dayMatch: String?, timeMatch: MatchResult?, relativeTimeMatch: MatchResult?) {
- val calendar = Calendar.getInstance()
-
- if (relativeTimeMatch != null) {
- // Parse the first time unit
- val firstValue = relativeTimeMatch.groupValues[1].takeIf { it.isNotEmpty() }?.toIntOrNull() ?: 0
- val halfValue = if (relativeTimeMatch.groupValues[2] == "half") 0.5 else 0.0 // Handle "half" scenario
- val firstUnit = relativeTimeMatch.groupValues[3].lowercase(Locale.ROOT)
-
- var secondValue = 0
- var secondUnit = ""
-
- // Check if a second time component exists (e.g., "30 minutes" in "3 hours and 30 minutes")
- if (relativeTimeMatch.groupValues[4].isNotEmpty()) {
- secondValue = relativeTimeMatch.groupValues[4].toInt()
- secondUnit = relativeTimeMatch.groupValues[5].lowercase(Locale.ROOT)
- }
-
- // Add first time unit
- when {
- firstUnit.contains("hour") || firstUnit.contains("hrs") -> {
- calendar.add(Calendar.HOUR_OF_DAY, firstValue) // Add full hours
- if (halfValue > 0) {
- calendar.add(Calendar.MINUTE, 30) // Correctly add 30 minutes for "half"
- }
- }
- firstUnit.contains("min") -> calendar.add(Calendar.MINUTE, firstValue)
- }
-
- // Add second time unit if available
- when {
- secondUnit.contains("hour") || secondUnit.contains("hrs") -> calendar.add(Calendar.HOUR_OF_DAY, secondValue)
- secondUnit.contains("minute") || secondUnit.contains("minutes") || secondUnit.contains("min") -> calendar.add(Calendar.MINUTE, secondValue)
- }
- }else if (timeMatch != null) {
- // Handle absolute time
- val hour = timeMatch.groupValues[1].toInt()
- val minutes = timeMatch.groupValues[3].toIntOrNull() ?: 0 // Default minutes to 0 if missing
- val amPm = timeMatch.groupValues[4].uppercase(Locale.ROOT)
-
- // Convert time to 24-hour format
- val hour24 = when {
- amPm == "PM" && hour != 12 -> hour + 12
- amPm == "AM" && hour == 12 -> 0
- // If no AM/PM, assume 24-hour format
- else -> hour
- }
-
- calendar.set(Calendar.HOUR_OF_DAY, hour24)
- calendar.set(Calendar.MINUTE, minutes)
- }
-
- val repeatDays = mutableListOf()
-
- // Adjust day and determine repeat days (existing logic)
- when {
- dayMatch == "today" -> { /* No change needed */ }
- dayMatch == "tomorrow" -> calendar.add(Calendar.DAY_OF_MONTH, 1)
- dayMatch == "next week" -> calendar.add(Calendar.DAY_OF_MONTH, 7)
- dayMatch == "next weekend" -> {
- val currentDayOfWeek = calendar.get(Calendar.DAY_OF_WEEK)
- val daysUntilSaturday = (Calendar.SATURDAY - currentDayOfWeek + 7) % 7
- calendar.add(Calendar.DAY_OF_MONTH, daysUntilSaturday)
- }
- dayMatch != null -> {
- val dayMap = mapOf(
- "sunday" to Calendar.SUNDAY, "monday" to Calendar.MONDAY,
- "tuesday" to Calendar.TUESDAY, "wednesday" to Calendar.WEDNESDAY,
- "thursday" to Calendar.THURSDAY, "friday" to Calendar.FRIDAY,
- "saturday" to Calendar.SATURDAY
- )
-
- dayMap.entries.forEach { (dayName, dayConstant) ->
- if (dayMatch.contains(dayName)) {
- repeatDays.add(dayConstant)
- }
- }
-
- if (repeatDays.isEmpty()) {
- val targetDay = dayMap.entries.firstOrNull { dayMatch.contains(it.key) }?.value
- if (targetDay != null) {
- val currentDayOfWeek = calendar.get(Calendar.DAY_OF_WEEK)
- val daysToAdd = (targetDay - currentDayOfWeek + 7) % 7
- calendar.add(Calendar.DAY_OF_MONTH, if (daysToAdd == 0) 7 else daysToAdd)
- }
- }
- }
- }
-
- // Create the intent to set the alarm
- val intent = Intent(AlarmClock.ACTION_SET_ALARM).apply {
- putExtra(AlarmClock.EXTRA_MESSAGE, "New Alarm")
- putExtra(AlarmClock.EXTRA_HOUR, calendar.get(Calendar.HOUR_OF_DAY))
- putExtra(AlarmClock.EXTRA_MINUTES, calendar.get(Calendar.MINUTE))
- putExtra(AlarmClock.EXTRA_SKIP_UI, true)
-
- if (repeatDays.isNotEmpty()) {
- putExtra(AlarmClock.EXTRA_DAYS, ArrayList(repeatDays))
- }
- addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
- }
-
- // Start the intent
- globalContext?.startActivity(intent)
- }
- //End Alarm
-
- //For Reminder
- private fun Reminder(itemId: Long, loadingItemId: Long, speak: Boolean, category: Category){
- try {
- if (chatList.isNotEmpty()) {
- val prompt = chatList.find { it.id == itemId }?.englishText?.replace("[.?!]+".toRegex(), "")?:return
-
- //val dayRegex = "(today|tomorrow|next week|next weekend|next month|((Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday)( morning| evening)?))".toRegex(RegexOption.IGNORE_CASE)
- val dayRegex = "(?i)\\b(today|tomorrow|next week|next weekend|((Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday)( morning| evening| night| afternoon)?))\\b".toRegex(RegexOption.IGNORE_CASE)
- //val timeRegex = "(\\d{1,2})(:(\\d{2}))?\\s*(AM|PM)?".toRegex(RegexOption.IGNORE_CASE)
- val timeRegex = "(?i)\\b(\\d{1,2})(:(\\d{2}))?\\s*(AM|PM)?\\b".toRegex(RegexOption.IGNORE_CASE)
- //val relativeTimeRegex = "(\\d+)\\s*(hours?|mins?|minutes?)\\s*(from now|later|next)?".toRegex(RegexOption.IGNORE_CASE)
- val relativeTimeRegex = "(?i)\\b(\\d+)?\\s*(?:and\\s*(half))?\\s*(hours?|hrs?|mins?|minutes?|seconds?|secs?)\\s*(?:and\\s*(\\d+))?\\s*(mins?|minutes?|secs?|seconds?)?\\s*(from now|later|next)?\\b".toRegex(RegexOption.IGNORE_CASE)
- val contextRegex = "remind me(?: to| for| of| about| to create)?\\s+(.*?)(?:\\s+(at|on|in)\\s+.*)?$".toRegex(RegexOption.IGNORE_CASE)
-
- val contextMatch = contextRegex.find(prompt)
- val context = contextMatch?.groupValues?.get(1)?.trim() ?: "Reminder"
-
- val timeMatch = timeRegex.find(prompt)
- val relativeTimeMatch = relativeTimeRegex.find(prompt)
- val dayMatch = dayRegex.find(prompt)?.value?.lowercase()
-
- if(timeMatch == null && relativeTimeMatch == null){
- viewModelScope.launch {
- lockState = LockState.LockReminder(day = dayMatch, context = context)
- processResponse(getRandomResponse(ResponseStrings.promptForTime), loadingItemId, speak, Category.OTHER)
- }
- }else{
- setReminderFromPrompt(dayMatch, timeMatch, relativeTimeMatch, context)
-
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.reminderSetSuccess), loadingItemId, speak, category)
- }
- }
-
- }
- }catch (e: Exception){
- Log.d("Exception", e.message.toString())
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.genericError), loadingItemId, speak, Category.OTHER)
- }
- }
- }
-
- private fun handleReminderLockState (itemId: Long, loadingItemId: Long, speak: Boolean, state: LockState.LockReminder){
- try{
- val prompt = chatList.find { it.id == itemId }?.englishText?.replace("[.?!]+".toRegex(), "")?:return
-
- val timeRegex = "(\\d{1,2})(:(\\d{2}))?\\s*(AM|PM)?".toRegex(RegexOption.IGNORE_CASE)
- val relativeTimeRegex = "(\\d+)\\s*(hours?|mins?|minutes?)\\s*(from now|later|next)?".toRegex(RegexOption.IGNORE_CASE)
-
- val context = state.context ?: "Reminder"
- val dayMatch = state.day
- val timeMatch = timeRegex.find(prompt)
- val relativeTimeMatch = relativeTimeRegex.find(prompt)
-
- if(timeMatch == null && relativeTimeMatch == null){
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.invalidTime), loadingItemId, speak, Category.OTHER)
- }
- return
- }else{
- setReminderFromPrompt(dayMatch, timeMatch, relativeTimeMatch, context)
- lockState = LockState.None
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.reminderSetSuccess), loadingItemId, speak, Category.ALARM)
- }
- }
- }catch (e: Exception){
- Log.d("Exception", e.message.toString())
- viewModelScope.launch {
- processResponse(getRandomResponse(ResponseStrings.genericError), loadingItemId, speak, Category.OTHER)
- }
- }
- }
-
- private fun setReminderFromPrompt(dayMatch: String?, timeMatch: MatchResult?, relativeTimeMatch: MatchResult?,context: String) {
- val calendar = Calendar.getInstance()
-
- if (relativeTimeMatch != null) {
- // Parse the first time unit
- val firstValue = relativeTimeMatch.groupValues[1].takeIf { it.isNotEmpty() }?.toIntOrNull() ?: 0
- val halfValue = if (relativeTimeMatch.groupValues[2] == "half") 0.5 else 0.0 // Handle "half" scenario
- val firstUnit = relativeTimeMatch.groupValues[3].lowercase(Locale.ROOT)
-
- var secondValue = 0
- var secondUnit = ""
-
- // Check if a second time component exists (e.g., "30 minutes" in "3 hours and 30 minutes")
- if (relativeTimeMatch.groupValues[4].isNotEmpty()) {
- secondValue = relativeTimeMatch.groupValues[4].toInt()
- secondUnit = relativeTimeMatch.groupValues[5].lowercase(Locale.ROOT)
- }
-
- // Add first time unit
- when {
- firstUnit.contains("hour") || firstUnit.contains("hrs") -> {
- calendar.add(Calendar.HOUR_OF_DAY, firstValue) // Add full hours
- if (halfValue > 0) {
- calendar.add(Calendar.MINUTE, 30) // Correctly add 30 minutes for "half"
- }
- }
- firstUnit.contains("min") -> calendar.add(Calendar.MINUTE, firstValue)
- }
-
- // Add second time unit if available
- when {
- secondUnit.contains("hour") || secondUnit.contains("hrs") -> calendar.add(Calendar.HOUR_OF_DAY, secondValue)
- secondUnit.contains("minute") || secondUnit.contains("minutes") || secondUnit.contains("min") -> calendar.add(Calendar.MINUTE, secondValue)
- }
- } else if (timeMatch != null) {
- // Handle absolute time
- val hour = timeMatch.groupValues[1].toInt()
- val minutes = timeMatch.groupValues[3].toIntOrNull() ?: 0 // Default minutes to 0 if missing
- val amPm = timeMatch.groupValues[4].uppercase(Locale.ROOT)
-
- // Convert time to 24-hour format
- val hour24 = when {
- amPm == "PM" && hour != 12 -> hour + 12
- amPm == "AM" && hour == 12 -> 0
- // If no AM/PM, assume 24-hour format
- else -> hour
- }
-
- calendar.set(Calendar.HOUR_OF_DAY, hour24)
- calendar.set(Calendar.MINUTE, minutes)
- }
-
- val repeatDays = mutableListOf()
-
- // Adjust day and determine repeat days (existing logic)
- when {
- dayMatch == "today" -> { /* No change needed */ }
- dayMatch == "tomorrow" -> calendar.add(Calendar.DAY_OF_MONTH, 1)
- dayMatch == "next week" -> calendar.add(Calendar.DAY_OF_MONTH, 7)
- dayMatch == "next weekend" -> {
- val currentDayOfWeek = calendar.get(Calendar.DAY_OF_WEEK)
- val daysUntilSaturday = (Calendar.SATURDAY - currentDayOfWeek + 7) % 7
- calendar.add(Calendar.DAY_OF_MONTH, daysUntilSaturday)
- }
- dayMatch != null -> {
- val dayMap = mapOf(
- "sunday" to Calendar.SUNDAY, "monday" to Calendar.MONDAY,
- "tuesday" to Calendar.TUESDAY, "wednesday" to Calendar.WEDNESDAY,
- "thursday" to Calendar.THURSDAY, "friday" to Calendar.FRIDAY,
- "saturday" to Calendar.SATURDAY
- )
-
- dayMap.entries.forEach { (dayName, dayConstant) ->
- if (dayMatch.contains(dayName)) {
- repeatDays.add(dayConstant)
- }
- }
-
- if (repeatDays.isEmpty()) {
- val targetDay = dayMap.entries.firstOrNull { dayMatch.contains(it.key) }?.value
- if (targetDay != null) {
- val currentDayOfWeek = calendar.get(Calendar.DAY_OF_WEEK)
- val daysToAdd = (targetDay - currentDayOfWeek + 7) % 7
- calendar.add(Calendar.DAY_OF_MONTH, if (daysToAdd == 0) 7 else daysToAdd)
- }
- }
- }
- }
-
- // Create the intent to set the alarm
- val intent = Intent(AlarmClock.ACTION_SET_ALARM).apply {
- putExtra(AlarmClock.EXTRA_MESSAGE, context)
- putExtra(AlarmClock.EXTRA_HOUR, calendar.get(Calendar.HOUR_OF_DAY))
- putExtra(AlarmClock.EXTRA_MINUTES, calendar.get(Calendar.MINUTE))
- putExtra(AlarmClock.EXTRA_SKIP_UI, true)
- putExtra(AlarmClock.EXTRA_RINGTONE, AlarmClock.VALUE_RINGTONE_SILENT)
-
- if (repeatDays.isNotEmpty()) {
- putExtra(AlarmClock.EXTRA_DAYS, ArrayList(repeatDays))
- }
- addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
- }
-
- // Start the intent
- globalContext?.startActivity(intent)
- }
- //End Reminder
-
- private fun getRandomResponse(responses: List): String {
- return responses.randomOrNull() ?: responses.firstOrNull() ?: "An unexpected error occurred." // Fallback
- }
-
- object ResponseStrings {
-
- // Permission for making call Related
- val permissionContactsCall = listOf(
- "I need permission to access your contacts and make phone call. Please allow and try again.",
- "Hey, I’ll need access to your contacts and calling first. Can you allow that?",
- "Looks like I don’t have permission to call yet. Please enable it and retry.",
- "I can help with that, but I need contacts and call access first.",
- "Please grant me contacts and calling permission so I can make the call for you.",
- "I can’t make calls without your permission. Could you turn it on?"
- )
-
- // Location permission for weather report
- val permissionLocation = listOf(
- "I need permission to access your location. Please try again.",
- "I can get the weather, but I’ll need your location first.",
- "Looks like location access isn’t granted. Please allow it and try again.",
- "Hey, could you enable location permission so I can show the weather?",
- "To fetch the forecast, I’ll need your location. Can you grant access?",
- "Without your location, I can’t check the weather. Please turn it on."
- )
-
- // Asking user to turn on location for weather report
- val locationServiceOff = listOf(
- "I need to know your location for that. Please turn on your location and try again.",
- "looks like location is off. Could you enable it?",
- "I can’t continue without your location. Please turn it on.",
- "Hey, I’ll need your location for this. Can you switch it on?",
- "Looks like your location services are disabled. Please activate them.",
- "Please turn on location, then I’ll be able to continue."
- )
-
- // Failure / Not Found Related
- val callFailed = listOf(
- "Sorry, failed to make call. Please try again.",
- "Oops, the call didn’t go through. Want to retry?",
- "I couldn’t complete the call. Please try again.",
- "Looks like that call failed. Give it another shot?",
- "Something went wrong with the call. Please try again later.",
- "I wasn’t able to connect the call. Can we try once more?"
- )
-
- // Contact not found
- val contactNotFound = listOf(
- "I cannot find such contact, please try again.",
- "I didn’t find that contact in your list.",
- "No contact matched that name, could you check and retry?",
- "Sorry, I couldn’t locate that person in your contacts.",
- "Looks like that name isn’t saved in your contacts.",
- "I couldn’t find that contact. Maybe try with a different name?"
- )
-
- // Song not found
- val songNotFound = listOf(
- "I can not find such song, please try again.",
- "Sorry, I couldn’t find that track.",
- "no song matched your request. Want to try another?",
- "I wasn’t able to locate that song. Please retry.",
- "Looks like that song isn’t available right now.",
- "I couldn’t find that one. Maybe try with a different title?"
- )
-
- // Location not found
- val locationNotFound = listOf(
- "I can not find such location, please try again.",
- "Sorry, I couldn’t figure out where that is.",
- "I wasn’t able to locate that place.",
- "No results for that location. Can you check and try again?",
- "Looks like that place isn’t on my map data.",
- "I couldn’t find that spot. Maybe try with a different name?"
- )
-
- // Weather report unavailable
- val weatherReportUnavailable = listOf(
- "Seems weather report is not available, please try again.",
- "Sorry, I couldn’t get the weather right now.",
- "The weather service isn’t responding. Please try later.",
- "Looks like weather data is down at the moment.",
- "I wasn’t able to fetch the forecast. Can you retry later?",
- "Weather info isn’t available right now. Please check back soon."
- )
-
- // Location unknown, suggest city
- val locationUnknownSuggestCity = listOf(
- "Your location is not available to me. Please try again with your city name.",
- "I couldn’t detect your location. Could you tell me your city instead?",
- "Looks like location services aren’t working. Please provide your city name.",
- "I’m not getting your location right now. Can you enter your city?",
- "Sorry, I can’t access your current location. A city name would help.",
- "Your location seems unavailable. Please try with your city name."
- )
-
- // Invalid time
- val invalidTime = listOf(
- "That doesn't seems like an actual time. Please try again.",
- "I didn’t recognize that as a valid time.",
- "That time format looks off. Could you retry?",
- "Sorry, I couldn’t understand that time input.",
- "That doesn’t look like a proper time. Please try again.",
- "Can you give me a valid time so I can continue?"
- )
-
- // Alarm success
- val alarmSetSuccess = listOf(
- "Alarm set successfully...",
- "Done! Your alarm is ready.",
- "Great, I’ve set the alarm for you.",
- "All set, your alarm has been scheduled.",
- "Alarm saved successfully.",
- "Okay, I’ve configured the alarm as requested."
- )
-
- // Reminder success
- val reminderSetSuccess = listOf(
- "Reminder set successfully...",
- "Done! Your reminder is ready.",
- "Great, I’ve saved the reminder for you.",
- "All set, your reminder has been scheduled.",
- "Reminder saved successfully.",
- "Okay, I’ve created the reminder as requested."
- )
-
- // Prompt for time
- val promptForTime = listOf(
- "Sure, at what time?",
- "Alright, when should I set it?",
- "Okay, what time would you like?",
- "Got it, please tell me the time.",
- "When do you want me to set it for?",
- "Sure thing, what time works for you?"
- )
-
- // Generic error
- val genericError = listOf(
- "Something went wrong, please try again.",
- "Oops, that didn’t work. Please retry.",
- "I ran into an issue. Can you try again?",
- "Sorry, something broke there. Please try once more.",
- "That didn’t go through. Could you retry?",
- "An error popped up. Let’s try that again."
- )
- }
-
- fun saveKeys(youtubeApiKey: String, chatApiKey: String) {
- viewModelScope.launch(Dispatchers.IO) {
- with(securedPreferences.edit()) {
- if (youtubeApiKey.isNotBlank()) {
- putString("youtube_api_key", youtubeApiKey)
- Log.d("MainViewModel", "YouTube API Key saved securely.")
- }
- if (chatApiKey.isNotBlank()) {
- putString("chat_api_key", chatApiKey)
- Log.d("MainViewModel", "Chat API Key saved securely.")
- }
- apply()
- }
- }
- }
-
- fun loadYoutubeKey(): String? {
- // First, try to retrieve the key from secured SharedPreferences
- var youtubeKey = securedPreferences.getString("youtube_api_key", null)
-
- // If the key is not in SharedPreferences, fall back to BuildConfig
- if (youtubeKey.isNullOrBlank()) {
- Log.d("MainViewModel", "YouTube key not found in SharedPreferences, checking BuildConfig.")
- youtubeKey = BuildConfig.YOUTUBE_API_KEY
- }
-
- return youtubeKey
- }
-
- fun loadChatKey(): String? {
- // First, try to retrieve the key from secured SharedPreferences
- var chatKey = securedPreferences.getString("chat_api_key", null)
-
- // If the key is not in SharedPreferences, fall back to BuildConfig
- if (chatKey.isNullOrBlank()) {
- Log.d("MainViewModel", "Chat key not found in SharedPreferences, checking BuildConfig.")
- chatKey = BuildConfig.GROQ_API_KEY
- }
-
- return chatKey
- }
-
}
diff --git a/app/src/main/java/com/app/assistant/viewmodel/MainViewModelActions.kt b/app/src/main/java/com/app/assistant/viewmodel/MainViewModelActions.kt
new file mode 100644
index 0000000..22fe72a
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/viewmodel/MainViewModelActions.kt
@@ -0,0 +1,323 @@
+package com.app.assistant.viewmodel
+
+import android.content.Intent
+import android.app.Application
+import androidx.lifecycle.viewModelScope
+import com.app.assistant.model.Conversation
+import com.app.assistant.util.Category
+import com.app.assistant.util.LockState
+import com.app.assistant.util.Constants.MAIN_CONTEXT
+import kotlinx.coroutines.launch
+import java.net.URI
+
+// Call Contact UseCase Delegate
+fun MainViewModel.callContact(
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ category: Category,
+) {
+ if (chatList.isNotEmpty()) {
+ val prompt = chatList.find { it.id == itemId }?.text ?: return
+ viewModelScope.launch {
+ callContactUseCase.execute(
+ prompt = prompt,
+ onPermissionRequest = { permissions ->
+ _uiEvent.send(UIEvent.RequestPermissions(permissions, 102))
+ processResponse(getRandomResponse(ResponseStrings.permissionContactsCall(getApplication())), loadingItemId, speak, Category.OTHER)
+ },
+ onIntentTriggered = { action ->
+ _uiEvent.send(UIEvent.StartIntent(action.toIntent()))
+ },
+ onSuccess = { name, dialUri ->
+ processResponse(
+ name,
+ loadingItemId,
+ false,
+ category,
+ navigationURI = dialUri
+ )
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+ }
+}
+
+// Play Song UseCase Delegate
+fun MainViewModel.playSong(
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ category: Category,
+) {
+ val prompt = chatList.find { it.id == itemId }?.text ?: return
+ viewModelScope.launch {
+ playSongUseCase.execute(
+ prompt = prompt,
+ onIntentTriggered = { action ->
+ _uiEvent.send(UIEvent.StartIntent(action.toIntent()))
+ },
+ onSuccess = { songName, videoId, thumbnailUrl, videoUri ->
+ processResponse(
+ getApplication().getString(com.app.assistant.R.string.playing_song, songName),
+ loadingItemId,
+ speak,
+ category,
+ thumbnailUrl,
+ videoUri
+ )
+ },
+ onMissingApiKey = { searchQuery ->
+ processResponse(getApplication().getString(com.app.assistant.R.string.youtube_key_missing), loadingItemId, speak, Category.OTHER)
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+}
+
+// Navigate UseCase Delegate
+fun MainViewModel.navigate(
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ category: Category,
+) {
+ val prompt = chatList.find { it.id == itemId }?.text ?: return
+ viewModelScope.launch {
+ navigateUseCase.execute(
+ prompt = prompt,
+ onIntentTriggered = { action ->
+ _uiEvent.send(UIEvent.StartIntent(action.toIntent()))
+ },
+ onSuccess = { location, navigationUri ->
+ processResponse(
+ getApplication().getString(com.app.assistant.R.string.navigating_to, location),
+ loadingItemId,
+ speak,
+ category,
+ navigationURI = navigationUri
+ )
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+}
+
+// Weather UseCase Delegate
+fun MainViewModel.fetchWeather(
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ category: Category,
+) {
+ val prompt = chatList.find { it.id == itemId }?.text ?: return
+ viewModelScope.launch {
+ getWeatherUseCase.execute(
+ prompt = prompt,
+ onPermissionRequest = { permissions ->
+ _uiEvent.send(UIEvent.RequestPermissions(permissions, 103))
+ processResponse(getRandomResponse(ResponseStrings.permissionLocation(getApplication())), loadingItemId, speak, Category.OTHER)
+ },
+ onLocationRequest = {
+ _uiEvent.send(
+ UIEvent.GetLocationForWeather(
+ itemId,
+ loadingItemId,
+ speak,
+ category.name,
+ prompt
+ )
+ )
+ },
+ onSuccess = { response, location ->
+ processResponse(
+ response,
+ loadingItemId,
+ speak,
+ category,
+ navigationURI = URI("https://www.google.com/search?q=weather+$location")
+ )
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+}
+
+fun MainViewModel.onLocationReceived(
+ lat: Double,
+ long: Double,
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ categoryName: String,
+ prompt: String,
+) {
+ viewModelScope.launch {
+ getWeatherUseCase.processLocationWeather(
+ lat = lat,
+ long = long,
+ prompt = prompt,
+ onSuccess = { response, city ->
+ processResponse(
+ response,
+ loadingItemId,
+ speak,
+ Category.valueOf(categoryName),
+ navigationURI = URI("https://www.google.com/search?q=weather+${
+ java.net.URLEncoder.encode(city, java.nio.charset.StandardCharsets.UTF_8.toString())
+ }")
+ )
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+}
+
+fun MainViewModel.onLocationFailed(
+ loadingItemId: Long,
+ speak: Boolean,
+ errorType: String,
+) {
+ viewModelScope.launch {
+ val responseText = when (errorType) {
+ "GPS_OFF" -> getRandomResponse(ResponseStrings.locationServiceOff(getApplication()))
+ "UNAVAILABLE" -> getRandomResponse(ResponseStrings.weatherReportUnavailable(getApplication()))
+ else -> getRandomResponse(ResponseStrings.locationUnknownSuggestCity(getApplication()))
+ }
+ processResponse(responseText, loadingItemId, speak, Category.OTHER)
+ }
+}
+
+// Set Alarm UseCase Delegate
+fun MainViewModel.setAlarm(
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ category: Category,
+) {
+ val prompt = chatList.find { it.id == itemId }?.text ?: return
+ viewModelScope.launch {
+ setAlarmUseCase.execute(
+ prompt = prompt,
+ onPromptForTime = { dayMatch ->
+ lockState = LockState.LockAlarm(day = dayMatch)
+ processResponse(
+ getRandomResponse(ResponseStrings.promptForTime(getApplication())),
+ loadingItemId,
+ speak,
+ Category.OTHER
+ )
+ },
+ onSuccess = { action ->
+ _uiEvent.send(UIEvent.StartIntent(action.toIntent()))
+ processResponse(
+ getRandomResponse(ResponseStrings.alarmSetSuccess(getApplication())),
+ loadingItemId,
+ speak,
+ category
+ )
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+}
+
+fun MainViewModel.handleAlarmLockState(
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ state: LockState.LockAlarm,
+) {
+ val prompt = chatList.find { it.id == itemId }?.text ?: return
+ viewModelScope.launch {
+ setAlarmUseCase.execute(
+ prompt = prompt,
+ dayOverride = state.day,
+ onPromptForTime = {
+ processResponse(getRandomResponse(ResponseStrings.invalidTime(getApplication())), loadingItemId, speak, Category.OTHER)
+ },
+ onSuccess = { action ->
+ lockState = LockState.None
+ _uiEvent.send(UIEvent.StartIntent(action.toIntent()))
+ processResponse(getRandomResponse(ResponseStrings.alarmSetSuccess(getApplication())), loadingItemId, speak, Category.ALARM)
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+}
+
+// Set Reminder UseCase Delegate
+fun MainViewModel.setReminder(
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ category: Category,
+) {
+ val prompt = chatList.find { it.id == itemId }?.text ?: return
+ viewModelScope.launch {
+ setReminderUseCase.execute(
+ prompt = prompt,
+ onPromptForTime = { dayMatch, context ->
+ lockState = LockState.LockReminder(day = dayMatch, context = context)
+ processResponse(getRandomResponse(ResponseStrings.promptForTime(getApplication())), loadingItemId, speak, Category.OTHER)
+ },
+ onSuccess = { action ->
+ _uiEvent.send(UIEvent.StartIntent(action.toIntent()))
+ processResponse(getRandomResponse(ResponseStrings.reminderSetSuccess(getApplication())), loadingItemId, speak, category)
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+}
+
+fun MainViewModel.handleReminderLockState(
+ itemId: Long,
+ loadingItemId: Long,
+ speak: Boolean,
+ state: LockState.LockReminder,
+) {
+ val prompt = chatList.find { it.id == itemId }?.text ?: return
+ viewModelScope.launch {
+ setReminderUseCase.execute(
+ prompt = prompt,
+ dayOverride = state.day,
+ contextOverride = state.context,
+ onPromptForTime = { _, _ ->
+ processResponse(getRandomResponse(ResponseStrings.invalidTime(getApplication())), loadingItemId, speak, Category.OTHER)
+ },
+ onSuccess = { action ->
+ lockState = LockState.None
+ _uiEvent.send(UIEvent.StartIntent(action.toIntent()))
+ processResponse(getRandomResponse(ResponseStrings.reminderSetSuccess(getApplication())), loadingItemId, speak, Category.ALARM)
+ },
+ onFailure = { errorMsg ->
+ processResponse(errorMsg, loadingItemId, speak, Category.OTHER)
+ }
+ )
+ }
+}
+
+fun MainViewModel.callAI(
+ loadingItemId: Long,
+ speak: Boolean,
+ category: Category,
+) {
+ this.callAI(loadingItemId, speak, category)
+}
diff --git a/app/src/main/java/com/app/assistant/viewmodel/MainViewModelFactory.kt b/app/src/main/java/com/app/assistant/viewmodel/MainViewModelFactory.kt
index e8d4535..a5f090f 100644
--- a/app/src/main/java/com/app/assistant/viewmodel/MainViewModelFactory.kt
+++ b/app/src/main/java/com/app/assistant/viewmodel/MainViewModelFactory.kt
@@ -3,17 +3,72 @@ package com.app.assistant.viewmodel
import android.app.Application
import androidx.lifecycle.ViewModel
import androidx.lifecycle.ViewModelProvider
+import com.app.assistant.db.DynamicConversationRepository
+import com.app.assistant.repository.ContactsRepository
+import com.app.assistant.repository.SettingsRepository
+import com.app.assistant.repository.WeatherRepository
+import com.app.assistant.usecase.CallContactUseCase
+import com.app.assistant.usecase.GetAiResponseUseCase
+import com.app.assistant.usecase.GetWeatherUseCase
+import com.app.assistant.usecase.NavigateUseCase
+import com.app.assistant.usecase.PlaySongUseCase
+import com.app.assistant.usecase.ProcessChatCommandUseCase
+import com.app.assistant.usecase.SetAlarmUseCase
+import com.app.assistant.usecase.SetReminderUseCase
+import okhttp3.OkHttpClient
+import java.util.concurrent.TimeUnit
class MainViewModelFactory(
private val application: Application,
- private val speak: Boolean
+ private val speak: Boolean,
) : ViewModelProvider.Factory {
-
override fun create(modelClass: Class): T {
if (modelClass.isAssignableFrom(MainViewModel::class.java)) {
+ val settingsRepo = SettingsRepository(application)
+ val contactsRepo = ContactsRepository(application)
+ val dbRepo = DynamicConversationRepository(application)
+
+ val okHttpClient = okHttpClient
+
+ val weatherRepo = WeatherRepository(application, okHttpClient)
+
+ val permissionChecker = AndroidPermissionChecker(application)
+ val resourceProvider = AndroidResourceProvider(application)
+
+ val getAiResponseUseCase = GetAiResponseUseCase(settingsRepo, okHttpClient)
+ val callContactUseCase = CallContactUseCase(permissionChecker, contactsRepo)
+ val playSongUseCase = PlaySongUseCase(resourceProvider, settingsRepo, okHttpClient)
+ val navigateUseCase = NavigateUseCase(resourceProvider)
+ val getWeatherUseCase = GetWeatherUseCase(permissionChecker, weatherRepo, getAiResponseUseCase)
+ val setAlarmUseCase = SetAlarmUseCase(resourceProvider)
+ val setReminderUseCase = SetReminderUseCase(resourceProvider)
+ val processChatCommandUseCase = ProcessChatCommandUseCase(getAiResponseUseCase, settingsRepo)
+
@Suppress("UNCHECKED_CAST")
- return MainViewModel(application, speak) as T
+ return MainViewModel(
+ application,
+ speak,
+ settingsRepo,
+ dbRepo,
+ callContactUseCase,
+ playSongUseCase,
+ navigateUseCase,
+ getWeatherUseCase,
+ setAlarmUseCase,
+ setReminderUseCase,
+ processChatCommandUseCase
+ ) as T
}
throw IllegalArgumentException("Unknown ViewModel class")
}
+
+ companion object {
+ val okHttpClient: OkHttpClient by lazy {
+ OkHttpClient.Builder()
+ .connectTimeout(60, TimeUnit.SECONDS)
+ .readTimeout(60, TimeUnit.SECONDS)
+ .writeTimeout(60, TimeUnit.SECONDS)
+ .build()
+ }
+ }
}
diff --git a/app/src/main/java/com/app/assistant/viewmodel/ResponseStrings.kt b/app/src/main/java/com/app/assistant/viewmodel/ResponseStrings.kt
new file mode 100644
index 0000000..bad5c56
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/viewmodel/ResponseStrings.kt
@@ -0,0 +1,51 @@
+package com.app.assistant.viewmodel
+
+import android.content.Context
+import com.app.assistant.R
+
+fun getRandomResponse(responses: List): String =
+ responses.randomOrNull() ?: responses.firstOrNull() ?: "An unexpected error occurred."
+
+object ResponseStrings {
+ fun permissionContactsCall(context: Context): List =
+ context.resources.getStringArray(R.array.permission_contacts_call).toList()
+
+ fun permissionLocation(context: Context): List =
+ context.resources.getStringArray(R.array.permission_location).toList()
+
+ fun locationServiceOff(context: Context): List =
+ context.resources.getStringArray(R.array.location_service_off).toList()
+
+ fun callFailed(context: Context): List =
+ context.resources.getStringArray(R.array.call_failed).toList()
+
+ fun contactNotFound(context: Context): List =
+ context.resources.getStringArray(R.array.contact_not_found).toList()
+
+ fun songNotFound(context: Context): List =
+ context.resources.getStringArray(R.array.song_not_found).toList()
+
+ fun locationNotFound(context: Context): List =
+ context.resources.getStringArray(R.array.location_not_found).toList()
+
+ fun weatherReportUnavailable(context: Context): List =
+ context.resources.getStringArray(R.array.weather_report_unavailable).toList()
+
+ fun locationUnknownSuggestCity(context: Context): List =
+ context.resources.getStringArray(R.array.location_unknown_suggest_city).toList()
+
+ fun invalidTime(context: Context): List =
+ context.resources.getStringArray(R.array.invalid_time).toList()
+
+ fun alarmSetSuccess(context: Context): List =
+ context.resources.getStringArray(R.array.alarm_set_success).toList()
+
+ fun reminderSetSuccess(context: Context): List =
+ context.resources.getStringArray(R.array.reminder_set_success).toList()
+
+ fun promptForTime(context: Context): List =
+ context.resources.getStringArray(R.array.prompt_for_time).toList()
+
+ fun genericError(context: Context): List =
+ context.resources.getStringArray(R.array.generic_error).toList()
+}
diff --git a/app/src/main/java/com/app/assistant/viewmodel/SettingsViewModel.kt b/app/src/main/java/com/app/assistant/viewmodel/SettingsViewModel.kt
new file mode 100644
index 0000000..b89e1e7
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/viewmodel/SettingsViewModel.kt
@@ -0,0 +1,559 @@
+package com.app.assistant.viewmodel
+
+import androidx.lifecycle.ViewModel
+import androidx.lifecycle.viewModelScope
+import com.app.assistant.BuildConfig
+import com.app.assistant.llm.LlmConfig
+import com.app.assistant.llm.LlmProvider
+import com.app.assistant.llm.ModelCapabilities
+import com.app.assistant.llm.ModelCapabilityProber
+import com.app.assistant.repository.SettingsRepository
+import kotlinx.coroutines.flow.MutableStateFlow
+import kotlinx.coroutines.flow.StateFlow
+import kotlinx.coroutines.flow.asStateFlow
+import kotlinx.coroutines.launch
+import okhttp3.OkHttpClient
+import kotlinx.coroutines.Dispatchers
+import java.io.File
+import android.util.Log
+import java.io.IOException
+import com.google.mlkit.common.model.DownloadConditions
+import com.google.mlkit.common.model.RemoteModelManager
+import com.google.mlkit.nl.translate.TranslateLanguage
+import com.google.mlkit.nl.translate.TranslateRemoteModel
+import java.lang.reflect.Modifier
+import android.content.Context
+import android.app.role.RoleManager
+import android.content.ComponentName
+import android.provider.Settings
+import android.os.Build
+
+sealed class DownloadState {
+ object Idle : DownloadState()
+ data class Downloading(val progress: Float, val currentBytes: Long, val totalBytes: Long) : DownloadState()
+ object Completed : DownloadState()
+ data class Error(val message: String) : DownloadState()
+}
+
+sealed class VerificationState {
+ object Idle : VerificationState()
+ object Verifying : VerificationState()
+ data class Success(val capabilities: ModelCapabilities) : VerificationState()
+ data class Error(val message: String) : VerificationState()
+}
+
+class SettingsViewModel(
+ private val settingsRepository: SettingsRepository,
+ private val okHttpClient: OkHttpClient
+) : ViewModel() {
+
+ private val modelManager = com.app.assistant.speech.SpeechModelManager(settingsRepository.context)
+ private val ttsModelManager = com.app.assistant.tts.TtsModelManager(settingsRepository.context)
+
+ private val _modelDownloadState = MutableStateFlow(DownloadState.Idle)
+ val modelDownloadState: StateFlow = _modelDownloadState.asStateFlow()
+
+ private val _isModelInstalled = MutableStateFlow(modelManager.isModelDownloaded())
+
+ private val _isTranslationEnabled = MutableStateFlow(settingsRepository.getIsTranslationEnabled())
+ val isTranslationEnabled: StateFlow = _isTranslationEnabled.asStateFlow()
+
+ private val _activeLanguageCode = MutableStateFlow(settingsRepository.getActiveLanguageCode())
+ val activeLanguageCode: StateFlow = _activeLanguageCode.asStateFlow()
+
+ private val _downloadedTranslationLanguages = MutableStateFlow>(emptySet())
+ val downloadedTranslationLanguages: StateFlow> = _downloadedTranslationLanguages.asStateFlow()
+
+ private val _downloadingTranslationLanguages = MutableStateFlow>(emptySet())
+ val downloadingTranslationLanguages: StateFlow> = _downloadingTranslationLanguages.asStateFlow()
+
+ var translationLanguages: List> = emptyList()
+ private set
+
+ private val _isDefaultAssistant = MutableStateFlow(false)
+ val isDefaultAssistant: StateFlow = _isDefaultAssistant.asStateFlow()
+
+ init {
+ translationLanguages = getPublicStaticFinalStringsWithNames(TranslateLanguage::class.java)
+ refreshDownloadedTranslationModels()
+ checkDefaultAssistantStatus()
+ }
+
+ fun checkDefaultAssistantStatus() {
+ val context = settingsRepository.context
+ val isDefault = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
+ val roleManager = context.getSystemService(Context.ROLE_SERVICE) as? RoleManager
+ roleManager?.isRoleHeld(RoleManager.ROLE_ASSISTANT) == true
+ } else {
+ val assistantSetting = Settings.Secure.getString(context.contentResolver, "assistant")
+ if (assistantSetting.isNullOrEmpty()) {
+ false
+ } else {
+ val currentAssistant = ComponentName.unflattenFromString(assistantSetting)
+ val myAssistant = ComponentName(context, com.app.assistant.AssistActivity::class.java)
+ currentAssistant == myAssistant
+ }
+ }
+ _isDefaultAssistant.value = isDefault
+ }
+
+ val isModelInstalled: StateFlow = _isModelInstalled.asStateFlow()
+
+ private val _ttsModelDownloadState = MutableStateFlow(DownloadState.Idle)
+ val ttsModelDownloadState: StateFlow = _ttsModelDownloadState.asStateFlow()
+
+ private val _isTtsModelInstalled = MutableStateFlow(ttsModelManager.isModelDownloaded())
+ val isTtsModelInstalled: StateFlow = _isTtsModelInstalled.asStateFlow()
+
+ private val _verificationState = MutableStateFlow(VerificationState.Idle)
+ val verificationState: StateFlow = _verificationState.asStateFlow()
+
+ private val _fetchedModels = MutableStateFlow>>(emptyMap())
+ val fetchedModels: StateFlow>> = _fetchedModels.asStateFlow()
+
+ private val _isFetchingModels = MutableStateFlow(false)
+ val isFetchingModels: StateFlow = _isFetchingModels.asStateFlow()
+
+ private val _modelFetchError = MutableStateFlow(null)
+ val modelFetchError: StateFlow = _modelFetchError.asStateFlow()
+
+ fun resetVerificationState() {
+ _verificationState.value = VerificationState.Idle
+ }
+
+ fun loadYoutubeKey(): String {
+ val key = settingsRepository.getYoutubeApiKey()
+ return if (key.isNullOrBlank()) BuildConfig.YOUTUBE_API_KEY else key
+ }
+
+ fun loadChatKey(): String {
+ val key = settingsRepository.getChatApiKey()
+ return if (key.isNullOrBlank()) BuildConfig.GROQ_API_KEY else key
+ }
+
+ fun loadEdgeTtsSubscriptionKey(): String {
+ val key = settingsRepository.getEdgeTtsSubscriptionKey()
+ return if (key.isNullOrBlank()) BuildConfig.EDGE_TTS_SUBSCRIPTION_KEY else key
+ }
+
+ fun loadLlmProvider(): LlmProvider {
+ val name = settingsRepository.getLlmProvider()
+ return try {
+ LlmProvider.valueOf(name)
+ } catch (e: Exception) {
+ LlmProvider.GROQ
+ }
+ }
+
+ fun loadLlmModel(): String = settingsRepository.getLlmModel()
+
+ fun loadLlmCustomUrl(): String = settingsRepository.getLlmCustomUrl()
+ fun loadLlmCustomHeaders(): String = settingsRepository.getLlmCustomHeaders()
+ fun loadLlmCustomResponsePath(): String = settingsRepository.getLlmCustomResponsePath()
+ fun loadLlmCustomRequestTemplate(): String = settingsRepository.getLlmCustomRequestTemplate()
+ fun loadLlmCustomMessageFormat(): String = settingsRepository.getLlmCustomMessageFormat()
+ fun loadLlmCustomSystemRole(): String = settingsRepository.getLlmCustomSystemRole()
+ fun loadLlmCustomUserRole(): String = settingsRepository.getLlmCustomUserRole()
+ fun loadLlmCustomAssistantRole(): String = settingsRepository.getLlmCustomAssistantRole()
+ fun loadUseLocalWhisper(): Boolean = settingsRepository.getUseLocalWhisper()
+ fun loadSttMode(): com.app.assistant.speech.SttMode = settingsRepository.getSttMode()
+
+ fun loadTtsMode(): com.app.assistant.tts.TtsMode = settingsRepository.getTtsMode()
+
+
+ fun deleteModel() {
+ viewModelScope.launch(Dispatchers.IO) {
+ modelManager.deleteLocalModel()
+ _isModelInstalled.value = false
+ val currentMode = settingsRepository.getSttMode()
+ if (currentMode != com.app.assistant.speech.SttMode.NATIVE) {
+ settingsRepository.setSttMode(com.app.assistant.speech.SttMode.NATIVE)
+ }
+ _modelDownloadState.value = DownloadState.Idle
+ }
+ }
+
+ fun cancelModelDownload() {
+ modelManager.cancelDownload()
+ _modelDownloadState.value = DownloadState.Idle
+ }
+
+ fun startModelDownload() {
+ viewModelScope.launch {
+ _modelDownloadState.value = DownloadState.Downloading(0f, 0L, modelManager.totalSizeBytes)
+ modelManager.downloadModel(
+ client = okHttpClient,
+ onProgress = { downloadedBytes, totalBytes ->
+ val progress = if (totalBytes > 0) downloadedBytes.toFloat() / totalBytes else 0f
+ _modelDownloadState.value = DownloadState.Downloading(progress, downloadedBytes, totalBytes)
+ },
+ onComplete = { success, errorMsg ->
+ viewModelScope.launch {
+ if (success) {
+ _modelDownloadState.value = DownloadState.Completed
+ _isModelInstalled.value = true
+ } else {
+ _modelDownloadState.value = DownloadState.Error(errorMsg ?: "Unknown download error")
+ _isModelInstalled.value = modelManager.isModelDownloaded()
+ }
+ }
+ }
+ )
+ }
+ }
+
+ fun startTtsModelDownload() {
+ viewModelScope.launch {
+ _ttsModelDownloadState.value = DownloadState.Downloading(0f, 0L, 0L)
+ ttsModelManager.downloadModel(
+ client = okHttpClient,
+ onProgress = { downloadedBytes, totalBytes ->
+ val progress = if (totalBytes > 0) downloadedBytes.toFloat() / totalBytes else 0f
+ _ttsModelDownloadState.value = DownloadState.Downloading(progress, downloadedBytes, totalBytes)
+ },
+ onComplete = { success, errorMsg ->
+ viewModelScope.launch {
+ if (success) {
+ _ttsModelDownloadState.value = DownloadState.Completed
+ _isTtsModelInstalled.value = true
+ } else {
+ _ttsModelDownloadState.value = DownloadState.Error(errorMsg ?: "Unknown download error")
+ _isTtsModelInstalled.value = ttsModelManager.isModelDownloaded()
+ }
+ }
+ }
+ )
+ }
+ }
+
+ fun cancelTtsModelDownload() {
+ ttsModelManager.cancelDownload()
+ _ttsModelDownloadState.value = DownloadState.Idle
+ }
+
+ fun deleteTtsModel() {
+ viewModelScope.launch(Dispatchers.IO) {
+ ttsModelManager.deleteLocalModel()
+ _isTtsModelInstalled.value = false
+ _ttsModelDownloadState.value = DownloadState.Idle
+ val currentMode = settingsRepository.getTtsMode()
+ if (currentMode != com.app.assistant.tts.TtsMode.NATIVE) {
+ settingsRepository.setTtsMode(com.app.assistant.tts.TtsMode.NATIVE)
+ }
+ }
+ }
+
+ fun saveSettings(
+ youtubeApiKey: String,
+ chatApiKey: String,
+ edgeTtsSubscriptionKey: String,
+ provider: LlmProvider,
+ model: String,
+ customUrl: String,
+ customHeaders: String,
+ customResponsePath: String,
+ customRequestTemplate: String,
+ customMessageFormat: String,
+ customSystemRole: String,
+ customUserRole: String,
+ customAssistantRole: String,
+ sttMode: com.app.assistant.speech.SttMode,
+ ttsMode: com.app.assistant.tts.TtsMode
+ ) {
+ settingsRepository.saveKeys(youtubeApiKey, chatApiKey, edgeTtsSubscriptionKey)
+ settingsRepository.setLlmProvider(provider.name)
+ settingsRepository.setLlmModel(model)
+ settingsRepository.setLlmCustomUrl(customUrl)
+ settingsRepository.setLlmCustomHeaders(customHeaders)
+ settingsRepository.setLlmCustomResponsePath(customResponsePath)
+ settingsRepository.setLlmCustomRequestTemplate(customRequestTemplate)
+ settingsRepository.setLlmCustomMessageFormat(customMessageFormat)
+ settingsRepository.setLlmCustomSystemRole(customSystemRole)
+ settingsRepository.setLlmCustomUserRole(customUserRole)
+ settingsRepository.setLlmCustomAssistantRole(customAssistantRole)
+ settingsRepository.setSttMode(sttMode)
+ settingsRepository.setTtsMode(ttsMode)
+ }
+
+ fun updateYoutubeApiKey(key: String) {
+ settingsRepository.saveKeys(
+ youtubeApiKey = key,
+ chatApiKey = settingsRepository.getChatApiKey() ?: "",
+ edgeTtsSubscriptionKey = settingsRepository.getEdgeTtsSubscriptionKey() ?: ""
+ )
+ }
+
+ fun updateChatApiKey(key: String) {
+ settingsRepository.saveKeys(
+ youtubeApiKey = settingsRepository.getYoutubeApiKey() ?: "",
+ chatApiKey = key,
+ edgeTtsSubscriptionKey = settingsRepository.getEdgeTtsSubscriptionKey() ?: ""
+ )
+ }
+
+ fun updateEdgeTtsSubscriptionKey(key: String) {
+ settingsRepository.saveKeys(
+ youtubeApiKey = settingsRepository.getYoutubeApiKey() ?: "",
+ chatApiKey = settingsRepository.getChatApiKey() ?: "",
+ edgeTtsSubscriptionKey = key
+ )
+ }
+
+ fun updateLlmProvider(provider: LlmProvider) {
+ settingsRepository.setLlmProvider(provider.name)
+ }
+
+ fun updateLlmModel(model: String) {
+ settingsRepository.setLlmModel(model)
+ }
+
+ fun updateLlmCustomUrl(url: String) {
+ settingsRepository.setLlmCustomUrl(url)
+ }
+
+ fun updateLlmCustomHeaders(headers: String) {
+ settingsRepository.setLlmCustomHeaders(headers)
+ }
+
+ fun updateLlmCustomResponsePath(path: String) {
+ settingsRepository.setLlmCustomResponsePath(path)
+ }
+
+ fun updateLlmCustomRequestTemplate(template: String) {
+ settingsRepository.setLlmCustomRequestTemplate(template)
+ }
+
+ fun updateLlmCustomMessageFormat(format: String) {
+ settingsRepository.setLlmCustomMessageFormat(format)
+ }
+
+ fun updateLlmCustomSystemRole(role: String) {
+ settingsRepository.setLlmCustomSystemRole(role)
+ }
+
+ fun updateLlmCustomUserRole(role: String) {
+ settingsRepository.setLlmCustomUserRole(role)
+ }
+
+ fun updateLlmCustomAssistantRole(role: String) {
+ settingsRepository.setLlmCustomAssistantRole(role)
+ }
+
+ fun updateSttMode(mode: com.app.assistant.speech.SttMode) {
+ settingsRepository.setSttMode(mode)
+ }
+
+ fun updateTtsMode(mode: com.app.assistant.tts.TtsMode) {
+ settingsRepository.setTtsMode(mode)
+ }
+
+
+
+ fun verifyModelAndSaveCapabilities(
+ provider: LlmProvider,
+ model: String,
+ apiKey: String,
+ customUrl: String,
+ customHeaders: String,
+ customResponsePath: String,
+ customRequestTemplate: String,
+ customMessageFormat: String,
+ customSystemRole: String,
+ customUserRole: String,
+ customAssistantRole: String
+ ) {
+ viewModelScope.launch {
+ _verificationState.value = VerificationState.Verifying
+
+ val config = if (provider == LlmProvider.CUSTOM) {
+ LlmConfig(
+ url = customUrl,
+ headers = parseHeadersJson(customHeaders),
+ responsePath = customResponsePath,
+ requestTemplate = customRequestTemplate,
+ messageFormat = customMessageFormat,
+ systemRole = customSystemRole.takeIf { it.isNotBlank() },
+ userRole = customUserRole,
+ assistantRole = customAssistantRole
+ )
+ } else {
+ provider.config
+ }
+
+ // 1. Test Text connection
+ val isTextOk = ModelCapabilityProber.probeTextConnection(
+ client = okHttpClient,
+ apiKey = apiKey,
+ model = model,
+ config = config
+ )
+
+ if (!isTextOk) {
+ _verificationState.value = VerificationState.Error("Connection check failed. Please verify API key, URL, or internet connection.")
+ settingsRepository.setIsModelVerified(false)
+ return@launch
+ }
+
+ // 2. Test Vision (Image) connection
+ val isVisionOk = ModelCapabilityProber.probeVisionConnection(
+ client = okHttpClient,
+ provider = provider,
+ apiKey = apiKey,
+ model = model,
+ config = config
+ )
+
+ // 3. Audio/Video checks (currently Gemini 1.5/2.0 is the main provider that supports audio/video input natively in the API)
+ val hasAudio = provider == LlmProvider.GEMINI && (model.contains("1.5") || model.contains("2.0") || model.contains("2.5"))
+ val hasVideo = provider == LlmProvider.GEMINI && (model.contains("1.5") || model.contains("2.0") || model.contains("2.5"))
+ val hasDoc = (provider == LlmProvider.GEMINI && (model.contains("1.5") || model.contains("2.0") || model.contains("2.5"))) || (provider == LlmProvider.ANTHROPIC && model.contains("claude-3"))
+
+ val capabilities = ModelCapabilities(
+ hasImageInput = isVisionOk,
+ hasAudioInput = hasAudio,
+ hasVideoInput = hasVideo,
+ hasDocumentInput = hasDoc
+ )
+
+ // Save detected capabilities in repository
+ settingsRepository.setIsImageSupported(isVisionOk)
+ settingsRepository.setIsAudioSupported(hasAudio)
+ settingsRepository.setIsVideoSupported(hasVideo)
+ settingsRepository.setIsDocumentSupported(hasDoc)
+ settingsRepository.setIsModelVerified(true)
+
+ _verificationState.value = VerificationState.Success(capabilities)
+
+ // Auto-fetch models on successful connection verification
+ fetchModelsForProvider(
+ provider = provider,
+ apiKey = apiKey,
+ customUrl = customUrl,
+ customHeaders = customHeaders
+ )
+ }
+ }
+
+ fun fetchModelsForProvider(
+ provider: LlmProvider,
+ apiKey: String,
+ customUrl: String = "",
+ customHeaders: String = ""
+ ) {
+ viewModelScope.launch {
+ if (apiKey.isBlank() && provider != LlmProvider.OLLAMA && provider != LlmProvider.OPEN_ROUTER) {
+ _modelFetchError.value = "API Key is required to fetch models."
+ return@launch
+ }
+ _isFetchingModels.value = true
+ _modelFetchError.value = null
+ try {
+ val models = ModelCapabilityProber.fetchAvailableModels(
+ client = okHttpClient,
+ provider = provider,
+ apiKey = apiKey,
+ customUrl = customUrl,
+ customHeaders = customHeaders
+ )
+ if (models.isNotEmpty()) {
+ _fetchedModels.value = _fetchedModels.value + (provider to models)
+ } else {
+ _modelFetchError.value = "No models returned from API."
+ }
+ } catch (e: Exception) {
+ _modelFetchError.value = "Failed to fetch models: ${e.message}"
+ } finally {
+ _isFetchingModels.value = false
+ }
+ }
+ }
+
+
+ private fun parseHeadersJson(jsonStr: String): Map {
+ return try {
+ val element = kotlinx.serialization.json.Json.parseToJsonElement(jsonStr)
+ if (element is kotlinx.serialization.json.JsonObject) {
+ element.mapValues { (_, value) ->
+ if (value is kotlinx.serialization.json.JsonPrimitive) value.content else value.toString()
+ }
+ } else {
+ emptyMap()
+ }
+ } catch (e: Exception) {
+ emptyMap()
+ }
+ }
+
+ fun updateTranslationEnabled(enabled: Boolean) {
+ _isTranslationEnabled.value = enabled
+ settingsRepository.setTranslationEnabled(enabled)
+ }
+
+ fun updateActiveLanguageCode(languageCode: String) {
+ _activeLanguageCode.value = languageCode
+ settingsRepository.setActiveLanguageCode(languageCode)
+ }
+
+ fun refreshDownloadedTranslationModels() {
+ RemoteModelManager.getInstance()
+ .getDownloadedModels(TranslateRemoteModel::class.java)
+ .addOnSuccessListener { downloadedModels ->
+ val langs = downloadedModels.map { it.language }.toSet()
+ _downloadedTranslationLanguages.value = langs
+ }
+ .addOnFailureListener { e ->
+ Log.e("SettingsViewModel", "Failed to fetch downloaded translation models: ${e.message}")
+ }
+ }
+
+ fun downloadTranslationModel(languageCode: String, onResult: (Boolean) -> Unit = {}) {
+ _downloadingTranslationLanguages.value = _downloadingTranslationLanguages.value + languageCode
+ val modelManager = RemoteModelManager.getInstance()
+ val languageModel = TranslateRemoteModel.Builder(languageCode).build()
+ val conditions = DownloadConditions.Builder().build()
+
+ modelManager.download(languageModel, conditions)
+ .addOnSuccessListener {
+ _downloadingTranslationLanguages.value = _downloadingTranslationLanguages.value - languageCode
+ refreshDownloadedTranslationModels()
+ onResult(true)
+ }
+ .addOnFailureListener { e ->
+ Log.e("SettingsViewModel", "Failed to download model for $languageCode: ${e.message}")
+ _downloadingTranslationLanguages.value = _downloadingTranslationLanguages.value - languageCode
+ onResult(false)
+ }
+ }
+
+ fun deleteTranslationModel(languageCode: String) {
+ val modelManager = RemoteModelManager.getInstance()
+ val languageModel = TranslateRemoteModel.Builder(languageCode).build()
+ modelManager.deleteDownloadedModel(languageModel)
+ .addOnSuccessListener {
+ refreshDownloadedTranslationModels()
+ }
+ .addOnFailureListener { e ->
+ Log.e("SettingsViewModel", "Failed to delete model for $languageCode: ${e.message}")
+ }
+ }
+
+ private fun getPublicStaticFinalStringsWithNames(clazz: Class<*>): List> {
+ val publicStaticFinalStringsWithNames = mutableListOf>()
+ val fields = clazz.declaredFields
+ for (field in fields) {
+ val modifiers = field.modifiers
+ if (Modifier.isPublic(modifiers) && Modifier.isStatic(modifiers) && Modifier.isFinal(modifiers)) {
+ if (field.type == String::class.java) {
+ try {
+ val name = field.name
+ val value = field.get(null) as String
+ publicStaticFinalStringsWithNames.add(Pair(name, value))
+ } catch (e: IllegalAccessException) {
+ e.printStackTrace()
+ }
+ }
+ }
+ }
+ return publicStaticFinalStringsWithNames
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/viewmodel/SettingsViewModelFactory.kt b/app/src/main/java/com/app/assistant/viewmodel/SettingsViewModelFactory.kt
new file mode 100644
index 0000000..dfc5367
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/viewmodel/SettingsViewModelFactory.kt
@@ -0,0 +1,19 @@
+package com.app.assistant.viewmodel
+
+import androidx.lifecycle.ViewModel
+import androidx.lifecycle.ViewModelProvider
+import com.app.assistant.repository.SettingsRepository
+import okhttp3.OkHttpClient
+
+class SettingsViewModelFactory(
+ private val settingsRepository: SettingsRepository,
+ private val okHttpClient: OkHttpClient
+) : ViewModelProvider.Factory {
+ override fun create(modelClass: Class): T {
+ if (modelClass.isAssignableFrom(SettingsViewModel::class.java)) {
+ @Suppress("UNCHECKED_CAST")
+ return SettingsViewModel(settingsRepository, okHttpClient) as T
+ }
+ throw IllegalArgumentException("Unknown ViewModel class")
+ }
+}
diff --git a/app/src/main/java/com/app/assistant/viewmodel/UIEvent.kt b/app/src/main/java/com/app/assistant/viewmodel/UIEvent.kt
new file mode 100644
index 0000000..0791a85
--- /dev/null
+++ b/app/src/main/java/com/app/assistant/viewmodel/UIEvent.kt
@@ -0,0 +1,42 @@
+package com.app.assistant.viewmodel
+
+import android.content.Intent
+import com.google.android.gms.common.api.ResolvableApiException
+
+sealed class UIEvent {
+ data class RequestPermissions(
+ val permissions: Array,
+ val requestCode: Int,
+ ) : UIEvent()
+
+ data class StartIntent(
+ val intent: Intent,
+ ) : UIEvent()
+
+ data class ShowToast(
+ val message: String,
+ ) : UIEvent()
+
+ data class SpeakText(
+ val text: String,
+ val queueMode: Int = 0,
+ ) : UIEvent()
+
+ object StopSpeaking : UIEvent()
+
+ object StartSpeechRecognition : UIEvent()
+
+ object StopSpeechRecognition : UIEvent()
+
+ data class GetLocationForWeather(
+ val itemId: Long,
+ val loadingItemId: Long,
+ val speak: Boolean,
+ val categoryName: String,
+ val prompt: String,
+ ) : UIEvent()
+
+ data class ResolveLocationSettings(
+ val exception: ResolvableApiException,
+ ) : UIEvent()
+}
diff --git a/app/src/main/res/drawable/bg_notification_button.xml b/app/src/main/res/drawable/bg_notification_button.xml
new file mode 100644
index 0000000..2b0c9f9
--- /dev/null
+++ b/app/src/main/res/drawable/bg_notification_button.xml
@@ -0,0 +1,6 @@
+
+
+
diff --git a/app/src/main/res/drawable/ic_arrow_drop_down.xml b/app/src/main/res/drawable/ic_arrow_drop_down.xml
new file mode 100644
index 0000000..da0c1a9
--- /dev/null
+++ b/app/src/main/res/drawable/ic_arrow_drop_down.xml
@@ -0,0 +1,9 @@
+
+
diff --git a/app/src/main/res/drawable/ic_attachment.xml b/app/src/main/res/drawable/ic_attachment.xml
new file mode 100644
index 0000000..fe04f24
--- /dev/null
+++ b/app/src/main/res/drawable/ic_attachment.xml
@@ -0,0 +1,10 @@
+
+
diff --git a/app/src/main/res/drawable/ic_back.xml b/app/src/main/res/drawable/ic_back.xml
new file mode 100644
index 0000000..1e635c5
--- /dev/null
+++ b/app/src/main/res/drawable/ic_back.xml
@@ -0,0 +1,9 @@
+
+
diff --git a/app/src/main/res/drawable/ic_camera.xml b/app/src/main/res/drawable/ic_camera.xml
new file mode 100644
index 0000000..b895d2d
--- /dev/null
+++ b/app/src/main/res/drawable/ic_camera.xml
@@ -0,0 +1,20 @@
+
+
+
+
diff --git a/app/src/main/res/drawable/ic_download.xml b/app/src/main/res/drawable/ic_download.xml
new file mode 100644
index 0000000..d4be3b2
--- /dev/null
+++ b/app/src/main/res/drawable/ic_download.xml
@@ -0,0 +1,9 @@
+
+
diff --git a/app/src/main/res/drawable/ic_fullscreen.xml b/app/src/main/res/drawable/ic_fullscreen.xml
new file mode 100644
index 0000000..d5f4b99
--- /dev/null
+++ b/app/src/main/res/drawable/ic_fullscreen.xml
@@ -0,0 +1,9 @@
+
+
diff --git a/app/src/main/res/drawable/ic_fullscreen_exit.xml b/app/src/main/res/drawable/ic_fullscreen_exit.xml
new file mode 100644
index 0000000..6592966
--- /dev/null
+++ b/app/src/main/res/drawable/ic_fullscreen_exit.xml
@@ -0,0 +1,9 @@
+
+
diff --git a/app/src/main/res/drawable/ic_live_modes.xml b/app/src/main/res/drawable/ic_live_modes.xml
new file mode 100644
index 0000000..2111f0a
--- /dev/null
+++ b/app/src/main/res/drawable/ic_live_modes.xml
@@ -0,0 +1,10 @@
+
+
diff --git a/app/src/main/res/drawable/ic_mic_off.xml b/app/src/main/res/drawable/ic_mic_off.xml
new file mode 100644
index 0000000..bd1ee8d
--- /dev/null
+++ b/app/src/main/res/drawable/ic_mic_off.xml
@@ -0,0 +1,20 @@
+
+
+
+
diff --git a/app/src/main/res/drawable/ic_qr_code_scanner.xml b/app/src/main/res/drawable/ic_qr_code_scanner.xml
new file mode 100644
index 0000000..2986c0c
--- /dev/null
+++ b/app/src/main/res/drawable/ic_qr_code_scanner.xml
@@ -0,0 +1,51 @@
+
+
+
diff --git a/app/src/main/res/drawable/ic_screen_share.xml b/app/src/main/res/drawable/ic_screen_share.xml
new file mode 100644
index 0000000..7f9016b
--- /dev/null
+++ b/app/src/main/res/drawable/ic_screen_share.xml
@@ -0,0 +1,32 @@
+
+
+
+
+
+
+
diff --git a/app/src/main/res/drawable/ic_speaker.xml b/app/src/main/res/drawable/ic_speaker.xml
new file mode 100644
index 0000000..b3ca905
--- /dev/null
+++ b/app/src/main/res/drawable/ic_speaker.xml
@@ -0,0 +1,10 @@
+
+
diff --git a/app/src/main/res/drawable/ic_stop.xml b/app/src/main/res/drawable/ic_stop.xml
index d787363..e88f949 100644
--- a/app/src/main/res/drawable/ic_stop.xml
+++ b/app/src/main/res/drawable/ic_stop.xml
@@ -1,5 +1,11 @@
-
-
-
+
+
diff --git a/app/src/main/res/drawable/ic_switch_camera.xml b/app/src/main/res/drawable/ic_switch_camera.xml
new file mode 100644
index 0000000..d3df457
--- /dev/null
+++ b/app/src/main/res/drawable/ic_switch_camera.xml
@@ -0,0 +1,24 @@
+
+
+
+
diff --git a/app/src/main/res/layout/notification_screen_capture_collapsed.xml b/app/src/main/res/layout/notification_screen_capture_collapsed.xml
new file mode 100644
index 0000000..dc72972
--- /dev/null
+++ b/app/src/main/res/layout/notification_screen_capture_collapsed.xml
@@ -0,0 +1,47 @@
+
+
+
+
+
+
+
+
diff --git a/app/src/main/res/layout/notification_screen_capture_expanded.xml b/app/src/main/res/layout/notification_screen_capture_expanded.xml
new file mode 100644
index 0000000..0ea50cd
--- /dev/null
+++ b/app/src/main/res/layout/notification_screen_capture_expanded.xml
@@ -0,0 +1,109 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml
index d907111..906b22c 100644
--- a/app/src/main/res/values/strings.xml
+++ b/app/src/main/res/values/strings.xml
@@ -1,3 +1,213 @@
Assistant
+ Settings
+ YouTube API Key
+ Edge TTS Subscription Key
+ Chat API Key
+ LLM Provider
+ Model Name
+ Custom Config (Markdown)
+ App Version: %1$s
+ Save
+ Cancel
+ Chats
+ Start new chat
+ Menu
+ Copy
+ Delete
+ Translate
+ Restart Chat
+ Chat in different languages
+ Search…
+ Are you sure you want to clear this chat from history?
+ Yes
+ No
+ Delete this message?
+
+
+ Navigation
+ Call
+ Type here...
+ Stop
+ Mic
+ Send
+
+
+ Downloading translation model.
+ Download completed, its recommended to use selected language keyboard.
+ Something went wrong, model download failed.
+
+
+ Playing %1$s
+ Your Youtube API key is missing or invalid.
+ Navigating to %1$s.
+ New Alarm
+ Reminder
+ I can not find such song, please try again.
+ I can not find such location, please try again.
+ Something went wrong, please try again.
+
+
+
+ - I need permission to access your contacts and make phone call. Please allow and try again.
+ - Hey, I’ll need access to your contacts and calling first. Can you allow that?
+ - Looks like I don’t have permission to call yet. Please enable it and retry.
+ - I can help with that, but I need contacts and call access first.
+ - Please grant me contacts and calling permission so I can make the call for you.
+ - I can’t make calls without your permission. Could you turn it on?
+
+
+
+ - I need permission to access your location. Please try again.
+ - I can get the weather, but I’ll need your location first.
+ - Looks like location access isn’t granted. Please allow it and try again.
+ - Hey, could you enable location permission so I can show the weather?
+ - To fetch the forecast, I’ll need your location. Can you grant access?
+ - Without your location, I can’t check the weather. Please turn it on.
+
+
+
+ - I need to know your location for that. Please turn on your location and try again.
+ - looks like location is off. Could you enable it?
+ - I can’t continue without your location. Please turn it on.
+ - Hey, I’ll need your location for this. Can you switch it on?
+ - Looks like your location services are disabled. Please activate them.
+ - Please turn on location, then I’ll be able to continue.
+
+
+
+ - Sorry, failed to make call. Please try again.
+ - Oops, the call didn’t go through. Want to retry?
+ - I couldn’t complete the call. Please try again.
+ - Looks like that call failed. Give it another shot?
+ - Something went wrong with the call. Please try again later.
+ - I wasn’t able to connect the call. Can we try once more?
+
+
+
+ - I cannot find such contact, please try again.
+ - I didn’t find that contact in your list.
+ - No contact matched that name, could you check and retry?
+ - Sorry, I couldn’t locate that person in your contacts.
+ - Looks like that name isn’t saved in your contacts.
+ - I couldn’t find that contact. Maybe try with a different name?
+
+
+
+ - I can not find such song, please try again.
+ - Sorry, I couldn’t find that track.
+ - no song matched your request. Want to try another?
+ - I wasn’t able to locate that song. Please retry.
+ - Looks like that song isn’t available right now.
+ - I couldn’t find that one. Maybe try with a different title?
+
+
+
+ - I can not find such location, please try again.
+ - Sorry, I couldn’t figure out where that is.
+ - I wasn’t able to locate that place.
+ - No results for that location. Can you check and try again?
+ - Looks like that place isn’t on my map data.
+ - I couldn’t find that spot. Maybe try with a different name?
+
+
+
+ - Seems weather report is not available, please try again.
+ - Sorry, I couldn’t get the weather right now.
+ - The weather service isn’t responding. Please try later.
+ - Looks like weather data is down at the moment.
+ - I wasn’t able to fetch the forecast. Can you retry later?
+ - Weather info isn’t available right now. Please check back soon.
+
+
+
+ - Your location is not available to me. Please try again with your city name.
+ - I couldn’t detect your location. Could you tell me your city instead?
+ - Looks like location services aren’t working. Please provide your city name.
+ - I’m not getting your location right now. Can you enter your city?
+ - Sorry, I can’t access your current location. A city name would help.
+ - Your location seems unavailable. Please try with your city name.
+
+
+
+ - That doesn\'t seems like an actual time. Please try again.
+ - I didn’t recognize that as a valid time.
+ - That time format looks off. Could you retry?
+ - Sorry, I couldn’t understand that time input.
+ - That doesn’t look like a proper time. Please try again.
+ - Can you give me a valid time so I can continue?
+
+
+
+ - Alarm set successfully...
+ - Done! Your alarm is ready.
+ - Great, I’ve set the alarm for you.
+ - All set, your alarm has been scheduled.
+ - Alarm saved successfully.
+ - Okay, I’ve configured the alarm as requested.
+
+
+
+ - Reminder set successfully...
+ - Done! Your reminder is ready.
+ - Great, I’ve saved the reminder for you.
+ - All set, your reminder has been scheduled.
+ - Reminder saved successfully.
+ - Okay, I’ve created the reminder as requested.
+
+
+
+ - Sure, at what time?
+ - Alright, when should I set it?
+ - Okay, what time would you like?
+ - Got it, please tell me the time.
+ - When do you want me to set it for?
+ - Sure thing, what time works for you?
+
+
+
+ - Something went wrong, please try again.
+ - Oops, that didn’t work. Please retry.
+ - I ran into an issue. Can you try again?
+ - Sorry, something broke there. Please try once more.
+ - That didn’t go through. Could you retry?
+ - An error popped up. Let’s try that again.
+
+
+
+ Speech to Text Settings
+ NVIDIA Parakeet Offline Model
+ The NVIDIA Parakeet TDT 0.6B v2 model is highly accurate but heavy (~660 MB). Downloading and using it will take time and storage.
+ Status: Installed (~660 MB)
+ Delete Model
+ Status: Not Downloaded
+ Download Model
+ Downloading… %1$d%% (%2$d MB / %3$d MB)
+ Download Completed!
+ Download Failed: %1$s
+ Clear
+ Retry
+ Use Native STT (Fast)
+ Uses the built-in system speech recognition. Fast and lightweight.
+ Use Only Parakeet Model (Offline)
+ Highly accurate, offline-first. May have a brief cold start time when opening the microphone.
+ Highly accurate, offline-first. Requires downloading the model first.
+ Use Hybrid Approach (Smart Switching)
+ Starts with Native STT instantly, loads Parakeet in the background, and switches automatically once loaded.
+ Starts with Native STT, loads Parakeet in background, and switches automatically once loaded. Requires model download.
+ Installed
+
+
+ Use STT API (Cloud)
+ Uses the cloud STT API (Groq Whisper) for highly accurate speech transcription.
+ Use TTS API (Cloud)
+ Uses the cloud TTS API (Edge-TTS) for high-quality natural voice generation.
+
+
+ Default Assistant App
+ Configure this app as your default system-wide voice and digital assistant.
+ Current Status: Default Assistant
+ Current Status: Not Default Assistant
+ Set as Default Assistant
+ Open Assistant Settings
\ No newline at end of file
diff --git a/app/src/main/res/xml/file_paths.xml b/app/src/main/res/xml/file_paths.xml
new file mode 100644
index 0000000..e5e18b5
--- /dev/null
+++ b/app/src/main/res/xml/file_paths.xml
@@ -0,0 +1,4 @@
+
+
+
diff --git a/build.gradle.kts b/build.gradle.kts
index f74b04b..d7f9313 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -2,4 +2,16 @@
plugins {
alias(libs.plugins.android.application) apply false
alias(libs.plugins.jetbrains.kotlin.android) apply false
-}
\ No newline at end of file
+ alias(libs.plugins.jetbrains.kotlin.serialization) apply false
+ alias(libs.plugins.spotless)
+}
+
+spotless {
+ kotlin {
+ target("**/*.kt")
+ ktlint()
+ trimTrailingWhitespace()
+ indentWithSpaces()
+ endWithNewline()
+ }
+}
diff --git a/docs/index.html b/docs/index.html
new file mode 100644
index 0000000..448eeb5
--- /dev/null
+++ b/docs/index.html
@@ -0,0 +1,717 @@
+
+
+
+ AI Assistant - Secure API Key QR Generator
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Client-Side Security
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Fill in one or more API keys and click "Generate QR" to preview
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Encoded Data Payload
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/qrcode.min.js b/docs/qrcode.min.js
new file mode 100644
index 0000000..993e88f
--- /dev/null
+++ b/docs/qrcode.min.js
@@ -0,0 +1 @@
+var QRCode;!function(){function a(a){this.mode=c.MODE_8BIT_BYTE,this.data=a,this.parsedData=[];for(var b=[],d=0,e=this.data.length;e>d;d++){var f=this.data.charCodeAt(d);f>65536?(b[0]=240|(1835008&f)>>>18,b[1]=128|(258048&f)>>>12,b[2]=128|(4032&f)>>>6,b[3]=128|63&f):f>2048?(b[0]=224|(61440&f)>>>12,b[1]=128|(4032&f)>>>6,b[2]=128|63&f):f>128?(b[0]=192|(1984&f)>>>6,b[1]=128|63&f):b[0]=f,this.parsedData=this.parsedData.concat(b)}this.parsedData.length!=this.data.length&&(this.parsedData.unshift(191),this.parsedData.unshift(187),this.parsedData.unshift(239))}function b(a,b){this.typeNumber=a,this.errorCorrectLevel=b,this.modules=null,this.moduleCount=0,this.dataCache=null,this.dataList=[]}function i(a,b){if(void 0==a.length)throw new Error(a.length+"/"+b);for(var c=0;c=f;f++){var h=0;switch(b){case d.L:h=l[f][0];break;case d.M:h=l[f][1];break;case d.Q:h=l[f][2];break;case d.H:h=l[f][3]}if(h>=e)break;c++}if(c>l.length)throw new Error("Too long data");return c}function s(a){var b=encodeURI(a).toString().replace(/\%[0-9a-fA-F]{2}/g,"a");return b.length+(b.length!=a?3:0)}a.prototype={getLength:function(){return this.parsedData.length},write:function(a){for(var b=0,c=this.parsedData.length;c>b;b++)a.put(this.parsedData[b],8)}},b.prototype={addData:function(b){var c=new a(b);this.dataList.push(c),this.dataCache=null},isDark:function(a,b){if(0>a||this.moduleCount<=a||0>b||this.moduleCount<=b)throw new Error(a+","+b);return this.modules[a][b]},getModuleCount:function(){return this.moduleCount},make:function(){this.makeImpl(!1,this.getBestMaskPattern())},makeImpl:function(a,c){this.moduleCount=4*this.typeNumber+17,this.modules=new Array(this.moduleCount);for(var d=0;d=7&&this.setupTypeNumber(a),null==this.dataCache&&(this.dataCache=b.createData(this.typeNumber,this.errorCorrectLevel,this.dataList)),this.mapData(this.dataCache,c)},setupPositionProbePattern:function(a,b){for(var c=-1;7>=c;c++)if(!(-1>=a+c||this.moduleCount<=a+c))for(var d=-1;7>=d;d++)-1>=b+d||this.moduleCount<=b+d||(this.modules[a+c][b+d]=c>=0&&6>=c&&(0==d||6==d)||d>=0&&6>=d&&(0==c||6==c)||c>=2&&4>=c&&d>=2&&4>=d?!0:!1)},getBestMaskPattern:function(){for(var a=0,b=0,c=0;8>c;c++){this.makeImpl(!0,c);var d=f.getLostPoint(this);(0==c||a>d)&&(a=d,b=c)}return b},createMovieClip:function(a,b,c){var d=a.createEmptyMovieClip(b,c),e=1;this.make();for(var f=0;f=g;g++)for(var h=-2;2>=h;h++)this.modules[d+g][e+h]=-2==g||2==g||-2==h||2==h||0==g&&0==h?!0:!1}},setupTypeNumber:function(a){for(var b=f.getBCHTypeNumber(this.typeNumber),c=0;18>c;c++){var d=!a&&1==(1&b>>c);this.modules[Math.floor(c/3)][c%3+this.moduleCount-8-3]=d}for(var c=0;18>c;c++){var d=!a&&1==(1&b>>c);this.modules[c%3+this.moduleCount-8-3][Math.floor(c/3)]=d}},setupTypeInfo:function(a,b){for(var c=this.errorCorrectLevel<<3|b,d=f.getBCHTypeInfo(c),e=0;15>e;e++){var g=!a&&1==(1&d>>e);6>e?this.modules[e][8]=g:8>e?this.modules[e+1][8]=g:this.modules[this.moduleCount-15+e][8]=g}for(var e=0;15>e;e++){var g=!a&&1==(1&d>>e);8>e?this.modules[8][this.moduleCount-e-1]=g:9>e?this.modules[8][15-e-1+1]=g:this.modules[8][15-e-1]=g}this.modules[this.moduleCount-8][8]=!a},mapData:function(a,b){for(var c=-1,d=this.moduleCount-1,e=7,g=0,h=this.moduleCount-1;h>0;h-=2)for(6==h&&h--;;){for(var i=0;2>i;i++)if(null==this.modules[d][h-i]){var j=!1;g>>e));var k=f.getMask(b,d,h-i);k&&(j=!j),this.modules[d][h-i]=j,e--,-1==e&&(g++,e=7)}if(d+=c,0>d||this.moduleCount<=d){d-=c,c=-c;break}}}},b.PAD0=236,b.PAD1=17,b.createData=function(a,c,d){for(var e=j.getRSBlocks(a,c),g=new k,h=0;h8*l)throw new Error("code length overflow. ("+g.getLengthInBits()+">"+8*l+")");for(g.getLengthInBits()+4<=8*l&&g.put(0,4);0!=g.getLengthInBits()%8;)g.putBit(!1);for(;;){if(g.getLengthInBits()>=8*l)break;if(g.put(b.PAD0,8),g.getLengthInBits()>=8*l)break;g.put(b.PAD1,8)}return b.createBytes(g,e)},b.createBytes=function(a,b){for(var c=0,d=0,e=0,g=new Array(b.length),h=new Array(b.length),j=0;j=0?p.get(q):0}}for(var r=0,m=0;mm;m++)for(var j=0;jm;m++)for(var j=0;j=0;)b^=f.G15<=0;)b^=f.G18<>>=1;return b},getPatternPosition:function(a){return f.PATTERN_POSITION_TABLE[a-1]},getMask:function(a,b,c){switch(a){case e.PATTERN000:return 0==(b+c)%2;case e.PATTERN001:return 0==b%2;case e.PATTERN010:return 0==c%3;case e.PATTERN011:return 0==(b+c)%3;case e.PATTERN100:return 0==(Math.floor(b/2)+Math.floor(c/3))%2;case e.PATTERN101:return 0==b*c%2+b*c%3;case e.PATTERN110:return 0==(b*c%2+b*c%3)%2;case e.PATTERN111:return 0==(b*c%3+(b+c)%2)%2;default:throw new Error("bad maskPattern:"+a)}},getErrorCorrectPolynomial:function(a){for(var b=new i([1],0),c=0;a>c;c++)b=b.multiply(new i([1,g.gexp(c)],0));return b},getLengthInBits:function(a,b){if(b>=1&&10>b)switch(a){case c.MODE_NUMBER:return 10;case c.MODE_ALPHA_NUM:return 9;case c.MODE_8BIT_BYTE:return 8;case c.MODE_KANJI:return 8;default:throw new Error("mode:"+a)}else if(27>b)switch(a){case c.MODE_NUMBER:return 12;case c.MODE_ALPHA_NUM:return 11;case c.MODE_8BIT_BYTE:return 16;case c.MODE_KANJI:return 10;default:throw new Error("mode:"+a)}else{if(!(41>b))throw new Error("type:"+b);switch(a){case c.MODE_NUMBER:return 14;case c.MODE_ALPHA_NUM:return 13;case c.MODE_8BIT_BYTE:return 16;case c.MODE_KANJI:return 12;default:throw new Error("mode:"+a)}}},getLostPoint:function(a){for(var b=a.getModuleCount(),c=0,d=0;b>d;d++)for(var e=0;b>e;e++){for(var f=0,g=a.isDark(d,e),h=-1;1>=h;h++)if(!(0>d+h||d+h>=b))for(var i=-1;1>=i;i++)0>e+i||e+i>=b||(0!=h||0!=i)&&g==a.isDark(d+h,e+i)&&f++;f>5&&(c+=3+f-5)}for(var d=0;b-1>d;d++)for(var e=0;b-1>e;e++){var j=0;a.isDark(d,e)&&j++,a.isDark(d+1,e)&&j++,a.isDark(d,e+1)&&j++,a.isDark(d+1,e+1)&&j++,(0==j||4==j)&&(c+=3)}for(var d=0;b>d;d++)for(var e=0;b-6>e;e++)a.isDark(d,e)&&!a.isDark(d,e+1)&&a.isDark(d,e+2)&&a.isDark(d,e+3)&&a.isDark(d,e+4)&&!a.isDark(d,e+5)&&a.isDark(d,e+6)&&(c+=40);for(var e=0;b>e;e++)for(var d=0;b-6>d;d++)a.isDark(d,e)&&!a.isDark(d+1,e)&&a.isDark(d+2,e)&&a.isDark(d+3,e)&&a.isDark(d+4,e)&&!a.isDark(d+5,e)&&a.isDark(d+6,e)&&(c+=40);for(var k=0,e=0;b>e;e++)for(var d=0;b>d;d++)a.isDark(d,e)&&k++;var l=Math.abs(100*k/b/b-50)/5;return c+=10*l}},g={glog:function(a){if(1>a)throw new Error("glog("+a+")");return g.LOG_TABLE[a]},gexp:function(a){for(;0>a;)a+=255;for(;a>=256;)a-=255;return g.EXP_TABLE[a]},EXP_TABLE:new Array(256),LOG_TABLE:new Array(256)},h=0;8>h;h++)g.EXP_TABLE[h]=1<h;h++)g.EXP_TABLE[h]=g.EXP_TABLE[h-4]^g.EXP_TABLE[h-5]^g.EXP_TABLE[h-6]^g.EXP_TABLE[h-8];for(var h=0;255>h;h++)g.LOG_TABLE[g.EXP_TABLE[h]]=h;i.prototype={get:function(a){return this.num[a]},getLength:function(){return this.num.length},multiply:function(a){for(var b=new Array(this.getLength()+a.getLength()-1),c=0;cf;f++)for(var g=c[3*f+0],h=c[3*f+1],i=c[3*f+2],k=0;g>k;k++)e.push(new j(h,i));return e},j.getRsBlockTable=function(a,b){switch(b){case d.L:return j.RS_BLOCK_TABLE[4*(a-1)+0];case d.M:return j.RS_BLOCK_TABLE[4*(a-1)+1];case d.Q:return j.RS_BLOCK_TABLE[4*(a-1)+2];case d.H:return j.RS_BLOCK_TABLE[4*(a-1)+3];default:return void 0}},k.prototype={get:function(a){var b=Math.floor(a/8);return 1==(1&this.buffer[b]>>>7-a%8)},put:function(a,b){for(var c=0;b>c;c++)this.putBit(1==(1&a>>>b-c-1))},getLengthInBits:function(){return this.length},putBit:function(a){var b=Math.floor(this.length/8);this.buffer.length<=b&&this.buffer.push(0),a&&(this.buffer[b]|=128>>>this.length%8),this.length++}};var l=[[17,14,11,7],[32,26,20,14],[53,42,32,24],[78,62,46,34],[106,84,60,44],[134,106,74,58],[154,122,86,64],[192,152,108,84],[230,180,130,98],[271,213,151,119],[321,251,177,137],[367,287,203,155],[425,331,241,177],[458,362,258,194],[520,412,292,220],[586,450,322,250],[644,504,364,280],[718,560,394,310],[792,624,442,338],[858,666,482,382],[929,711,509,403],[1003,779,565,439],[1091,857,611,461],[1171,911,661,511],[1273,997,715,535],[1367,1059,751,593],[1465,1125,805,625],[1528,1190,868,658],[1628,1264,908,698],[1732,1370,982,742],[1840,1452,1030,790],[1952,1538,1112,842],[2068,1628,1168,898],[2188,1722,1228,958],[2303,1809,1283,983],[2431,1911,1351,1051],[2563,1989,1423,1093],[2699,2099,1499,1139],[2809,2213,1579,1219],[2953,2331,1663,1273]],o=function(){var a=function(a,b){this._el=a,this._htOption=b};return a.prototype.draw=function(a){function g(a,b){var c=document.createElementNS("http://www.w3.org/2000/svg",a);for(var d in b)b.hasOwnProperty(d)&&c.setAttribute(d,b[d]);return c}var b=this._htOption,c=this._el,d=a.getModuleCount();Math.floor(b.width/d),Math.floor(b.height/d),this.clear();var h=g("svg",{viewBox:"0 0 "+String(d)+" "+String(d),width:"100%",height:"100%",fill:b.colorLight});h.setAttributeNS("http://www.w3.org/2000/xmlns/","xmlns:xlink","http://www.w3.org/1999/xlink"),c.appendChild(h),h.appendChild(g("rect",{fill:b.colorDark,width:"1",height:"1",id:"template"}));for(var i=0;d>i;i++)for(var j=0;d>j;j++)if(a.isDark(i,j)){var k=g("use",{x:String(i),y:String(j)});k.setAttributeNS("http://www.w3.org/1999/xlink","href","#template"),h.appendChild(k)}},a.prototype.clear=function(){for(;this._el.hasChildNodes();)this._el.removeChild(this._el.lastChild)},a}(),p="svg"===document.documentElement.tagName.toLowerCase(),q=p?o:m()?function(){function a(){this._elImage.src=this._elCanvas.toDataURL("image/png"),this._elImage.style.display="block",this._elCanvas.style.display="none"}function d(a,b){var c=this;if(c._fFail=b,c._fSuccess=a,null===c._bSupportDataURI){var d=document.createElement("img"),e=function(){c._bSupportDataURI=!1,c._fFail&&_fFail.call(c)},f=function(){c._bSupportDataURI=!0,c._fSuccess&&c._fSuccess.call(c)};return d.onabort=e,d.onerror=e,d.onload=f,d.src="data:image/gif;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==",void 0}c._bSupportDataURI===!0&&c._fSuccess?c._fSuccess.call(c):c._bSupportDataURI===!1&&c._fFail&&c._fFail.call(c)}if(this._android&&this._android<=2.1){var b=1/window.devicePixelRatio,c=CanvasRenderingContext2D.prototype.drawImage;CanvasRenderingContext2D.prototype.drawImage=function(a,d,e,f,g,h,i,j){if("nodeName"in a&&/img/i.test(a.nodeName))for(var l=arguments.length-1;l>=1;l--)arguments[l]=arguments[l]*b;else"undefined"==typeof j&&(arguments[1]*=b,arguments[2]*=b,arguments[3]*=b,arguments[4]*=b);c.apply(this,arguments)}}var e=function(a,b){this._bIsPainted=!1,this._android=n(),this._htOption=b,this._elCanvas=document.createElement("canvas"),this._elCanvas.width=b.width,this._elCanvas.height=b.height,a.appendChild(this._elCanvas),this._el=a,this._oContext=this._elCanvas.getContext("2d"),this._bIsPainted=!1,this._elImage=document.createElement("img"),this._elImage.style.display="none",this._el.appendChild(this._elImage),this._bSupportDataURI=null};return e.prototype.draw=function(a){var b=this._elImage,c=this._oContext,d=this._htOption,e=a.getModuleCount(),f=d.width/e,g=d.height/e,h=Math.round(f),i=Math.round(g);b.style.display="none",this.clear();for(var j=0;e>j;j++)for(var k=0;e>k;k++){var l=a.isDark(j,k),m=k*f,n=j*g;c.strokeStyle=l?d.colorDark:d.colorLight,c.lineWidth=1,c.fillStyle=l?d.colorDark:d.colorLight,c.fillRect(m,n,f,g),c.strokeRect(Math.floor(m)+.5,Math.floor(n)+.5,h,i),c.strokeRect(Math.ceil(m)-.5,Math.ceil(n)-.5,h,i)}this._bIsPainted=!0},e.prototype.makeImage=function(){this._bIsPainted&&d.call(this,a)},e.prototype.isPainted=function(){return this._bIsPainted},e.prototype.clear=function(){this._oContext.clearRect(0,0,this._elCanvas.width,this._elCanvas.height),this._bIsPainted=!1},e.prototype.round=function(a){return a?Math.floor(1e3*a)/1e3:a},e}():function(){var a=function(a,b){this._el=a,this._htOption=b};return a.prototype.draw=function(a){for(var b=this._htOption,c=this._el,d=a.getModuleCount(),e=Math.floor(b.width/d),f=Math.floor(b.height/d),g=[''],h=0;d>h;h++){g.push("");for(var i=0;d>i;i++)g.push(' ")}g.push("
"),c.innerHTML=g.join("");var j=c.childNodes[0],k=(b.width-j.offsetWidth)/2,l=(b.height-j.offsetHeight)/2;k>0&&l>0&&(j.style.margin=l+"px "+k+"px")},a.prototype.clear=function(){this._el.innerHTML=""},a}();QRCode=function(a,b){if(this._htOption={width:256,height:256,typeNumber:4,colorDark:"#000000",colorLight:"#ffffff",correctLevel:d.H},"string"==typeof b&&(b={text:b}),b)for(var c in b)this._htOption[c]=b[c];"string"==typeof a&&(a=document.getElementById(a)),this._android=n(),this._el=a,this._oQRCode=null,this._oDrawing=new q(this._el,this._htOption),this._htOption.text&&this.makeCode(this._htOption.text)},QRCode.prototype.makeCode=function(a){this._oQRCode=new b(r(a,this._htOption.correctLevel),this._htOption.correctLevel),this._oQRCode.addData(a),this._oQRCode.make(),this._el.title=a,this._oDrawing.draw(this._oQRCode),this.makeImage()},QRCode.prototype.makeImage=function(){"function"==typeof this._oDrawing.makeImage&&(!this._android||this._android>=3)&&this._oDrawing.makeImage()},QRCode.prototype.clear=function(){this._oDrawing.clear()},QRCode.CorrectLevel=d}();
\ No newline at end of file
diff --git a/gradle.properties b/gradle.properties
index 20e2a01..9a74fe4 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -6,7 +6,7 @@
# http://www.gradle.org/docs/current/userguide/build_environment.html
# Specifies the JVM arguments used for the daemon process.
# The setting is particularly useful for tweaking memory settings.
-org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8
+org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8 --add-opens=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
# When configured, Gradle will run in incubating parallel mode.
# This option should only be used with decoupled projects. For more details, visit
# https://developer.android.com/r/tools/gradle-multi-project-decoupled-projects
@@ -20,4 +20,6 @@ kotlin.code.style=official
# Enables namespacing of each library's R class so that its R class includes only the
# resources declared in the library itself and none from the library's dependencies,
# thereby reducing the size of the R class for that library
-android.nonTransitiveRClass=true
\ No newline at end of file
+android.nonTransitiveRClass=true
+kotlin.daemon.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8 --add-opens=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
+kotlin.compiler.execution.strategy=in-process
\ No newline at end of file
diff --git a/gradle/gradle-daemon-jvm.properties b/gradle/gradle-daemon-jvm.properties
new file mode 100644
index 0000000..5c34300
--- /dev/null
+++ b/gradle/gradle-daemon-jvm.properties
@@ -0,0 +1,13 @@
+#This file is generated by updateDaemonJvm
+toolchainUrl.FREE_BSD.AARCH64=https\://api.foojay.io/disco/v3.0/ids/56a19bc915b9ba2eb62ba7554c61b919/redirect
+toolchainUrl.FREE_BSD.X86_64=https\://api.foojay.io/disco/v3.0/ids/398ffe3949748bfb1d5636f023d228fd/redirect
+toolchainUrl.LINUX.AARCH64=https\://api.foojay.io/disco/v3.0/ids/56a19bc915b9ba2eb62ba7554c61b919/redirect
+toolchainUrl.LINUX.X86_64=https\://api.foojay.io/disco/v3.0/ids/398ffe3949748bfb1d5636f023d228fd/redirect
+toolchainUrl.MAC_OS.AARCH64=https\://api.foojay.io/disco/v3.0/ids/e99bae143b75f9a10ead10248f02055e/redirect
+toolchainUrl.MAC_OS.X86_64=https\://api.foojay.io/disco/v3.0/ids/04e088f8677de3b384108493cc9481d0/redirect
+toolchainUrl.UNIX.AARCH64=https\://api.foojay.io/disco/v3.0/ids/56a19bc915b9ba2eb62ba7554c61b919/redirect
+toolchainUrl.UNIX.X86_64=https\://api.foojay.io/disco/v3.0/ids/398ffe3949748bfb1d5636f023d228fd/redirect
+toolchainUrl.WINDOWS.AARCH64=https\://api.foojay.io/disco/v3.0/ids/e55dccbfe27cb97945148c61a39c89c5/redirect
+toolchainUrl.WINDOWS.X86_64=https\://api.foojay.io/disco/v3.0/ids/dbd05c4936d573642f94cd149e1356c8/redirect
+toolchainVendor=JETBRAINS
+toolchainVersion=21
diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml
index 5e81161..ecab12f 100644
--- a/gradle/libs.versions.toml
+++ b/gradle/libs.versions.toml
@@ -13,6 +13,22 @@ runtimeLivedata = "1.7.4"
playServicesLocation = "21.3.0"
uiTestAndroid = "1.7.0"
runtimeSaveable = "1.9.2"
+material3 = "1.2.1"
+material3Adaptive = "1.3.0-beta02"
+okhttp = "4.12.0"
+kotlinxSerializationJson = "1.6.3"
+mlkitTranslate = "17.0.3"
+commonmark = "0.24.0"
+coil = "3.0.0-rc02"
+securityCrypto = "1.1.0"
+mediapipeTasksText = "0.10.29"
+kotlinSerialization = "1.9.0"
+androidxComposeCompiler = "1.5.1"
+spotless = "8.4.0"
+room = "2.6.1"
+camerax = "1.4.0"
+mlkitBarcodeScanning = "17.3.0"
+
[libraries]
androidx-core-ktx = { group = "androidx.core", name = "core-ktx", version.ref = "coreKtx" }
@@ -28,14 +44,35 @@ androidx-ui-tooling = { group = "androidx.compose.ui", name = "ui-tooling" }
androidx-ui-tooling-preview = { group = "androidx.compose.ui", name = "ui-tooling-preview" }
androidx-ui-test-manifest = { group = "androidx.compose.ui", name = "ui-test-manifest" }
androidx-ui-test-junit4 = { group = "androidx.compose.ui", name = "ui-test-junit4" }
-androidx-material3 = { group = "androidx.compose.material3", name = "material3" }
+androidx-material3 = { group = "androidx.compose.material3", name = "material3", version.ref = "material3" }
+androidx-material3-window-size = { group = "androidx.compose.material3", name = "material3-window-size-class", version.ref = "material3" }
+androidx-material3-adaptive-navigation-suite = { group = "androidx.compose.material3", name = "material3-adaptive-navigation-suite", version.ref = "material3Adaptive" }
+okhttp = { group = "com.squareup.okhttp3", name = "okhttp", version.ref = "okhttp" }
+kotlinx-serialization-json = { group = "org.jetbrains.kotlinx", name = "kotlinx-serialization-json", version.ref = "kotlinxSerializationJson" }
+mlkit-translate = { group = "com.google.mlkit", name = "translate", version.ref = "mlkitTranslate" }
+mlkit-barcode-scanning = { group = "com.google.mlkit", name = "barcode-scanning", version.ref = "mlkitBarcodeScanning" }
+commonmark = { group = "org.commonmark", name = "commonmark", version.ref = "commonmark" }
+coil-compose = { group = "io.coil-kt.coil3", name = "coil-compose", version.ref = "coil" }
+coil-network-okhttp = { group = "io.coil-kt.coil3", name = "coil-network-okhttp", version.ref = "coil" }
+kotlin-reflect = { group = "org.jetbrains.kotlin", name = "kotlin-reflect", version.ref = "kotlin" }
androidx-datastore-core-android = { group = "androidx.datastore", name = "datastore-core-android", version.ref = "datastoreCoreAndroid" }
androidx-runtime-livedata = { group = "androidx.compose.runtime", name = "runtime-livedata", version.ref = "runtimeLivedata" }
play-services-location = { group = "com.google.android.gms", name = "play-services-location", version.ref = "playServicesLocation" }
androidx-ui-test-android = { group = "androidx.compose.ui", name = "ui-test-android", version.ref = "uiTestAndroid" }
+androidx-security-crypto = { group = "androidx.security", name = "security-crypto", version.ref = "securityCrypto" }
androidx-runtime-saveable = { group = "androidx.compose.runtime", name = "runtime-saveable", version.ref = "runtimeSaveable" }
+mediapipe-tasks-text = { group = "com.google.mediapipe", name = "tasks-text", version.ref = "mediapipeTasksText" }
+androidx-room-runtime = { group = "androidx.room", name = "room-runtime", version.ref = "room" }
+androidx-room-compiler = { group = "androidx.room", name = "room-compiler", version.ref = "room" }
+androidx-room-ktx = { group = "androidx.room", name = "room-ktx", version.ref = "room" }
+androidx-camera-core = { group = "androidx.camera", name = "camera-core", version.ref = "camerax" }
+androidx-camera-camera2 = { group = "androidx.camera", name = "camera-camera2", version.ref = "camerax" }
+androidx-camera-lifecycle = { group = "androidx.camera", name = "camera-lifecycle", version.ref = "camerax" }
+androidx-camera-view = { group = "androidx.camera", name = "camera-view", version.ref = "camerax" }
+
[plugins]
android-application = { id = "com.android.application", version.ref = "agp" }
jetbrains-kotlin-android = { id = "org.jetbrains.kotlin.android", version.ref = "kotlin" }
-
+jetbrains-kotlin-serialization = { id = "org.jetbrains.kotlin.plugin.serialization", version.ref = "kotlinSerialization" }
+spotless = { id = "com.diffplug.spotless", version.ref = "spotless" }
diff --git a/scratch/SettingsViewModel.kt b/scratch/SettingsViewModel.kt
new file mode 100644
index 0000000..f6fc21b
--- /dev/null
+++ b/scratch/SettingsViewModel.kt
@@ -0,0 +1,182 @@
+package com.app.assistant.viewmodel
+
+import androidx.lifecycle.ViewModel
+import androidx.lifecycle.viewModelScope
+import com.app.assistant.BuildConfig
+import com.app.assistant.llm.LlmConfig
+import com.app.assistant.llm.LlmProvider
+import com.app.assistant.llm.ModelCapabilities
+import com.app.assistant.llm.ModelCapabilityProber
+import com.app.assistant.repository.SettingsRepository
+import kotlinx.coroutines.flow.MutableStateFlow
+import kotlinx.coroutines.flow.StateFlow
+import kotlinx.coroutines.flow.asStateFlow
+import kotlinx.coroutines.launch
+import okhttp3.OkHttpClient
+
+sealed class VerificationState {
+ object Idle : VerificationState()
+ object Verifying : VerificationState()
+ data class Success(val capabilities: ModelCapabilities) : VerificationState()
+ data class Error(val message: String) : VerificationState()
+}
+
+class SettingsViewModel(
+ private val settingsRepository: SettingsRepository,
+ private val okHttpClient: OkHttpClient
+) : ViewModel() {
+
+ private val _verificationState = MutableStateFlow(VerificationState.Idle)
+ val verificationState: StateFlow = _verificationState.asStateFlow()
+
+ fun resetVerificationState() {
+ _verificationState.value = VerificationState.Idle
+ }
+
+ fun loadYoutubeKey(): String {
+ val key = settingsRepository.getYoutubeApiKey()
+ return if (key.isNullOrBlank()) BuildConfig.YOUTUBE_API_KEY else key
+ }
+
+ fun loadChatKey(): String {
+ val key = settingsRepository.getChatApiKey()
+ return if (key.isNullOrBlank()) BuildConfig.GROQ_API_KEY else key
+ }
+
+ fun loadLlmProvider(): LlmProvider {
+ val name = settingsRepository.getLlmProvider()
+ return try {
+ LlmProvider.valueOf(name)
+ } catch (e: Exception) {
+ LlmProvider.GROQ
+ }
+ }
+
+ fun loadLlmModel(): String = settingsRepository.getLlmModel()
+
+ fun loadLlmCustomUrl(): String = settingsRepository.getLlmCustomUrl()
+ fun loadLlmCustomHeaders(): String = settingsRepository.getLlmCustomHeaders()
+ fun loadLlmCustomResponsePath(): String = settingsRepository.getLlmCustomResponsePath()
+ fun loadLlmCustomRequestTemplate(): String = settingsRepository.getLlmCustomRequestTemplate()
+ fun loadLlmCustomMessageFormat(): String = settingsRepository.getLlmCustomMessageFormat()
+ fun loadLlmCustomSystemRole(): String = settingsRepository.getLlmCustomSystemRole()
+ fun loadLlmCustomUserRole(): String = settingsRepository.getLlmCustomUserRole()
+ fun loadLlmCustomAssistantRole(): String = settingsRepository.getLlmCustomAssistantRole()
+
+ fun saveSettings(
+ youtubeApiKey: String,
+ chatApiKey: String,
+ provider: LlmProvider,
+ model: String,
+ customUrl: String,
+ customHeaders: String,
+ customResponsePath: String,
+ customRequestTemplate: String,
+ customMessageFormat: String,
+ customSystemRole: String,
+ customUserRole: String,
+ customAssistantRole: String
+ ) {
+ settingsRepository.saveKeys(youtubeApiKey, chatApiKey)
+ settingsRepository.setLlmProvider(provider.name)
+ settingsRepository.setLlmModel(model)
+ settingsRepository.setLlmCustomUrl(customUrl)
+ settingsRepository.setLlmCustomHeaders(customHeaders)
+ settingsRepository.setLlmCustomResponsePath(customResponsePath)
+ settingsRepository.setLlmCustomRequestTemplate(customRequestTemplate)
+ settingsRepository.setLlmCustomMessageFormat(customMessageFormat)
+ settingsRepository.setLlmCustomSystemRole(customSystemRole)
+ settingsRepository.setLlmCustomUserRole(customUserRole)
+ settingsRepository.setLlmCustomAssistantRole(customAssistantRole)
+ }
+
+ fun verifyModelAndSaveCapabilities(
+ provider: LlmProvider,
+ model: String,
+ apiKey: String,
+ customUrl: String,
+ customHeaders: String,
+ customResponsePath: String,
+ customRequestTemplate: String,
+ customMessageFormat: String,
+ customSystemRole: String,
+ customUserRole: String,
+ customAssistantRole: String
+ ) {
+ viewModelScope.launch {
+ _verificationState.value = VerificationState.Verifying
+
+ val config = if (provider == LlmProvider.CUSTOM) {
+ LlmConfig(
+ url = customUrl,
+ headers = parseHeadersJson(customHeaders),
+ responsePath = customResponsePath,
+ requestTemplate = customRequestTemplate,
+ messageFormat = customMessageFormat,
+ systemRole = customSystemRole.takeIf { it.isNotBlank() },
+ userRole = customUserRole,
+ assistantRole = customAssistantRole
+ )
+ } else {
+ provider.config
+ }
+
+ // 1. Test Text connection
+ val isTextOk = ModelCapabilityProber.probeTextConnection(
+ client = okHttpClient,
+ provider = provider,
+ apiKey = apiKey,
+ model = model,
+ config = config
+ )
+
+ if (!isTextOk) {
+ _verificationState.value = VerificationState.Error("Connection check failed. Please verify API key, URL, or internet connection.")
+ settingsRepository.setIsModelVerified(false)
+ return@launch
+ }
+
+ // 2. Test Vision (Image) connection
+ val isVisionOk = ModelCapabilityProber.probeVisionConnection(
+ client = okHttpClient,
+ provider = provider,
+ apiKey = apiKey,
+ model = model,
+ config = config
+ )
+
+ // 3. Audio/Video checks (currently Gemini 1.5/2.0 is the main provider that supports audio/video input natively in the API)
+ val hasAudio = provider == LlmProvider.GEMINI && (model.contains("1.5") || model.contains("2.0"))
+ val hasVideo = provider == LlmProvider.GEMINI && (model.contains("1.5") || model.contains("2.0"))
+
+ val capabilities = ModelCapabilities(
+ hasImageInput = isVisionOk,
+ hasAudioInput = hasAudio,
+ hasVideoInput = hasVideo
+ )
+
+ // Save detected capabilities in repository
+ settingsRepository.setIsImageSupported(isVisionOk)
+ settingsRepository.setIsAudioSupported(hasAudio)
+ settingsRepository.setIsVideoSupported(hasVideo)
+ settingsRepository.setIsModelVerified(true)
+
+ _verificationState.value = VerificationState.Success(capabilities)
+ }
+ }
+
+ private fun parseHeadersJson(jsonStr: String): Map {
+ return try {
+ val element = kotlinx.serialization.json.Json.parseToJsonElement(jsonStr)
+ if (element is kotlinx.serialization.json.JsonObject) {
+ element.mapValues { (_, value) ->
+ if (value is kotlinx.serialization.json.JsonPrimitive) value.content else value.toString()
+ }
+ } else {
+ emptyMap()
+ }
+ } catch (e: Exception) {
+ emptyMap()
+ }
+ }
+}
diff --git a/screenshots/gif1.gif b/screenshots/gif1.gif
index 79e56f6..66acd6d 100644
Binary files a/screenshots/gif1.gif and b/screenshots/gif1.gif differ
diff --git a/screenshots/image1.png b/screenshots/image1.png
index 7f658ca..794c3f1 100644
Binary files a/screenshots/image1.png and b/screenshots/image1.png differ
diff --git a/screenshots/image2.png b/screenshots/image2.png
index ad1ac94..245a374 100644
Binary files a/screenshots/image2.png and b/screenshots/image2.png differ
diff --git a/screenshots/image3.png b/screenshots/image3.png
index 7c75c40..6be7390 100644
Binary files a/screenshots/image3.png and b/screenshots/image3.png differ
diff --git a/settings.gradle.kts b/settings.gradle.kts
index 3ac0e0f..01addf6 100644
--- a/settings.gradle.kts
+++ b/settings.gradle.kts
@@ -11,6 +11,9 @@ pluginManagement {
gradlePluginPortal()
}
}
+plugins {
+ id("org.gradle.toolchains.foojay-resolver-convention") version "0.10.0"
+}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
+
