How can I join https://github.com/Microsoft/Cognitive-Vision-Android with https://github.com/Azure-Samples/Cognitive-Speech-TTS so that the result of the analyzed image is read out by the text to speech API. As I was looking to make an app which combined both technologies to help blind people familiarise themselves with certain things by letting them know what it is.

I have the solution but don't know how I get it to work.

Analyze step 1: Add the event handler code for the form button

The analyzeImageButtonActionPerformed event handler method clears the form, displays the image specified in the URL, then calls the AnalyzeImage method to analyze the image. When AnalyzeImage returns, the method displays the formatted JSON response in the Response text area, extracts the first caption from the JSONObject, and displays the caption and the confidence level that the caption is correct.

private void analyzeImageButtonActionPerformed(java.awt.event.ActionEvent evt) { URL analyzeImageUrl;

// Clear out the previous image, response, and caption, if any. analyzeImage.setIcon(new ImageIcon()); analyzeCaptionLabel.setText(""); analyzeResponseTextArea.setText("");

// Display the image specified in the text box. try { analyzeImageUrl = new URL(analyzeImageUriTextBox.getText()); BufferedImage bImage = ImageIO.read(analyzeImageUrl); scaleAndShowImage(bImage, analyzeImage); } catch(IOException e) { analyzeResponseTextArea.setText("Error loading Analyze image: " + e.getMessage()); return; }

// Analyze the image. JSONObject jsonObj = AnalyzeImage(analyzeImageUrl.toString());

// A return of null indicates failure. if (jsonObj == null) { return; }

// Format and display the JSON response. analyzeResponseTextArea.setText(jsonObj.toString(2));

// Extract the text and confidence from the first caption in the description object. if (jsonObj.has("description") && jsonObj.getJSONObject("description").has("captions")) {

JSONObject jsonCaption = jsonObj.getJSONObject("description").getJSONArray("captions").getJSONObject(0);

if (jsonCaption.has("text") && jsonCaption.has("confidence")) {

analyzeCaptionLabel.setText("Caption: " + jsonCaption.getString("text") + " (confidence: " + jsonCaption.getDouble("confidence") + ")."); } } }

Analyze step 2: Add the wrapper for the REST API call

The AnalyzeImage method wraps the REST API call to analyze an image. The method returns a JSONObject describing the image, or null if there was an error.

private JSONObject AnalyzeImage(String imageUrl) { try (CloseableHttpClient httpclient = HttpClientBuilder.create().build()) { // Create the URI to access the REST API call for Analyze Image. String uriString = uriBasePreRegion + String.valueOf(subscriptionRegionComboBox.getSelectedItem()) + uriBasePostRegion + uriBaseAnalyze; URIBuilder builder = new URIBuilder(uriString);

// Request parameters. All of them are optional. builder.setParameter("visualFeatures", "Categories,Description,Color,Adult"); builder.setParameter("language", "en");

// Prepare the URI for the REST API call. URI uri = builder.build(); HttpPost request = new HttpPost(uri);

// Request headers. request.setHeader("Content-Type", "application/json"); request.setHeader("Ocp-Apim-Subscription-Key", subscriptionKeyTextField.getText());

// Request body. StringEntity reqEntity = new StringEntity("{\"url\":\"" + imageUrl + "\"}"); request.setEntity(reqEntity);

// Execute the REST API call and get the response entity. HttpResponse response = httpclient.execute(request); HttpEntity entity = response.getEntity();

// If we got a response, parse it and display it. if (entity != null) { // Return the JSONObject. String jsonString = EntityUtils.toString(entity); return new JSONObject(jsonString); } else { // No response. Return null. return null; } } catch (Exception e) { // Display error message. System.out.println(e.getMessage()); return null; } }

Analyze step 3: Run the application

Press F6 to run the application. Put your subscription key into the Subscription Key field and verify that you are using the correct region in Subscription Region. Enter a URL to an image to analyze, then click the Analyze Image button to analyze an image and see the result.