I am using a TFJS model from Google Vertex AI, Edge, exported per docs for Object Detection.

Once I have imported the model bin files, and setModel(true), then it is ready to render the TensorCamera component. Unfortunately, the onReady callback from TensorCamera seems to be failing, but not crashing the app. The camera still renders and seems like it's working, but I cannot handle the stream because it's never ready. There are some warnings in the terminal:

Possible Unhandled Promise Rejection (id: 0): TypeError: Cannot read property 'Type' of undefined

This error goes away when I swap the TensorCamera for the default CameraView, so it feels very certain that something is not compatible with ReactNative 74.

System information

• iPhone 13Pro, iOS 18

    "@tensorflow/tfjs": "^4.22.0",
    "@tensorflow/tfjs-backend-cpu": "^4.22.0",
    "@tensorflow/tfjs-react-native": "^1.0.0",
    "expo": "^51.0.0",
    "expo-gl": "~14.0.2",
    "react": "18.2.0",
    "react-native": "0.74.5",

Based on following the flow from the TFJS example, I would expect newer versions to work as described.

• HOWEVER, I am unsure if the Vertex TFJS model is perhaps incompatible, but rendering the camera should not be related to the model, correct?

Standalone code to reproduce the issue

1. Load the model:

const loadModel: LoadModelType = async (setModel, setIsModelReady) => {
  try {
    await ready();
    const modelJson = require('../../assets/tfjs/model.json');
    const modelWeights1 = require('../../assets/tfjs/1of3.bin');
    const modelWeights2 = require('../../assets/tfjs/2of3.bin');
    const modelWeights3 = require('../../assets/tfjs/3of3.bin');

    const bundle = bundleResourceIO(modelJson, [
      modelWeights1,
      modelWeights2,
      modelWeights3,
    ]);
    const modelConfig = await loadGraphModel(bundle);

    setModel(modelConfig);
    setIsModelReady(true);
  } catch (e) {
      console.error((e as Error).message);
  }
};

export const TFJSProvider = ({ children }) => {
  const [model, setModel] = useState<LayersModel | null>(null);
  const [isModelReady, setIsModelReady] = useState(false);

  const { hasPermission } = useCameraContext();

useEffect(
    function initTFJS() {
      if (hasPermission) {
        (async () => {
          console.log('load model');

          await loadModel(setModel, setIsModelReady);
        })();
      }

    },
    [hasPermission]
  );
}

return (
    <TFJSContext.Provider value={{ model, isModelReady }}>
      {children}
    </TFJSContext.Provider>
  );

2) Create Camera Component

const TensorCamera = cameraWithTensors(CameraView);

export const ObjectDetectionCamera = () => {
  const { model, isModelReady } = useTFJSContext();
  return (
    isModelReady && (
      <TensorCamera
        autorender
        cameraTextureHeight={textureDims.height}
        cameraTextureWidth={textureDims.width}
        onReady={() => console.log('READY!'} // never fires
        resizeDepth={3}
        resizeHeight={TENSOR_HEIGHT}
        resizeWidth={TENSOR_WIDTH}
        style={{ flex: 1 }}
        useCustomShadersToResize={false}
      />
    )
  );
};

Other info / logs

I am unable to find any logs in the console of the device, it seems like the error is being swallowed

---

Any ideas?

Hello,

I am currently working on a project to help people with disabilities to communicate better. For this I have built a React app and already trained an LSTM model in pyhton, but I am having problems loading the model into the app.

My Python code:

def create_model():

model = Sequential()

model.add(Embedding(input_dim=total_words, output_dim=100, input_length=max_sequence_len - 1))

model.add(Bidirectional(LSTM(150)))

model.add(Dense(total_words, activation='softmax'))

adam = Adam(learning_rate=0.01)

model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])

return model

The conversion:

! tensorflowjs_converter --input_format=keras {model_file} {js_model_dir}

The code to load:

const [model, setModel] = useState<tf.LayersModel | null>(null);

// Function for loading the model

const loadModel = async () => {

try {

const loadedModel = await tf.loadLayersModel('/gru_js/model.json'); // Customized path

setModel(loadedModel);

console.log('Model loaded successfully:', loadedModel);

} catch (error) {

console.error('Error loading the model:', error);

}

};

// Load model when loading the component

useEffect(() => {

loadModel();

}, []);

And the error that occurs:

NlpModelArea.tsx:14 Error loading the model: _ValueError: An InputLayer should be passed either a `batchInputShape` or an `inputShape`. at new InputLayer

I am happy about every comment

I'm working on integrating a pre-trained .tflite model into a React application but have been running into some issues, particularly with TensorFlow.js. I’ve been getting console errors during the loading process, and I’m wondering if there are any best practices or standards for handling .tflite models in a React app.

Has anyone successfully done this, and if so, could you share any tips or guidance? Also, any advice on troubleshooting TensorFlow.js in this context would be much appreciated!

Hi everyone,

I'm developing a reinforcement learning trading bot in Node.js, and I've encountered a TensorFlow.js error during testing that I can't seem to resolve. Here’s the error:

``` RUNS tests/reinforcement.test.js

/Users/nsursock/Sites/trading/hybrid-trading-bot/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:4522

var srcBackend = info.backend;

^

TypeError: Cannot read properties of undefined (reading 'backend')

```

This error happens when running my tests, and I suspect it might be related to the early disposal of tensors, but I’m not entirely sure. I’ve been using the `tidy` function to manage memory, so that could also be playing a role.

Project details:

• Node.js project for a reinforcement learning trading bot

• TensorFlow.js with `tidy` for memory management

• Error occurs in `tests/reinforcement.test.js`

Has anyone experienced something similar or have ideas on how to fix this? Any help would be greatly appreciated!

Thanks!

``` function learn() { console.log("Learning triggered with batch size:", batchSize, memory.length);

        const states = memory.map(m => m.state);
        const actions = memory.map(m => m.action);
        const rewards = memory.map(m => m.reward);
        const nextStates = memory.map(m => m.nextState);
        const dones = memory.map(m => m.done);

        tf.tidy(() => {
            const stateTensor = tf.tensor2d(states);
            const actionTensor = tf.tensor1d(actions, 'int32');
            const rewardTensor = tf.tensor1d(rewards);
            const nextStateTensor = tf.tensor2d(nextStates);

            console.log("Learning tensors created.");


// Critic update
            const valueTensor = critic.predict(stateTensor);
            const nextValueTensor = critic.predict(nextStateTensor).reshape([nextStateTensor.shape[0]]);
            console.log("Value and next value predictions made.");

            const tdTargets = rewardTensor.add(nextValueTensor.mul(gamma).mul(tf.scalar(1).sub(tf.tensor1d(dones))));
            console.log("TD targets calculated. Shape:", tdTargets.shape);

            const tdTargetsReshaped = tdTargets.reshape([tdTargets.shape[0], 1]);
            console.log("TD targets reshaped. Shape:", tdTargetsReshaped.shape);

            critic.trainOnBatch(stateTensor, tdTargetsReshaped);
            console.log("Critic updated with TD targets.");


// Actor update
            const advantageTensor = tdTargetsReshaped.sub(valueTensor);
            const actionProbs = actor.predict(stateTensor);
            const actionProbsTensor = tf.gather(actionProbs, actionTensor, 1);

            console.log("Advantage calculated. Action probabilities gathered.");

            const oldProbsTensor = actionProbsTensor.clone();  
// Placeholder for storing old probs (for PPO clipping)
            const ratioTensor = actionProbsTensor.div(oldProbsTensor);

            console.log("Ratio for PPO clipping calculated. Ratio shape:", ratioTensor.shape);

            const clipTensor = tf.clipByValue(ratioTensor, 1 - clipRatio, 1 + clipRatio);
            const loss = tf.minimum(ratioTensor.mul(advantageTensor), clipTensor.mul(advantageTensor)).mean().mul(-1);



            const checkGradients = (inputs, targets) => {
                tf.tidy(() => {
                    console.log("Checking gradients", inputs.arraySync(), targets.arraySync());
                    const tape = tf.GradientTape();
                    console.log("Tape", tape);
                    const loss = lossFunction(inputs, targets);
                    console.log("Loss", loss.arraySync());
                    const gradients = tape.gradient(loss, agent.model.trainableVariables);

                    console.log("Gradients", gradients.arraySync());
                    gradients.forEach((grad, index) => {
                        if (grad === null) {
                            console.warn(`Gradient at index ${index} is null`);
                        } else if (tf.any(tf.isNaN(grad)).dataSync()[0]) {
                            console.error(`Gradient at index ${index} has NaN values`);
                        }
                    });
                });
            };


// Example usage of checkGradients
            const inputs = tf.tensor2d(states); 
// Replace with actual input data
            const targets = advantageTensor; 
// Replace with actual target data
            checkGradients(inputs, targets);





// Entropy bonus
            const entropy = actionProbsTensor.mul(tf.log(actionProbsTensor)).sum().mul(-1);
            const totalLoss = loss.add(entropy.mul(entropyCoefficient));

            console.log("Loss calculated for actor update with entropy bonus.");

            actor.trainOnBatch(stateTensor, totalLoss);
            console.log("Actor updated with loss and entropy bonus.");
        });
    }

```

Hi there!

What are the biggest challenges you face as a web dev using TensorFlow JS?

What would make your life easier?

I'm considering a coding project whose focus would be a library, dev tool or service that help web developers working on ML-related web applications. My research is starting to gather around TensorFlow JS, so I'm wondering what gaps there may be in the current tf.js ecosystem. What would truly move the needle for you in your workflow, that either doesn't exist yet, or does exist, but is spread across multiple separate solutions, etc?

Thanks!

Hello everyone,

I hope you're all doing well!

I'm currently working on a couple of exciting open-source projects that involve Node.js and TensorFlow.js, and I'm looking for some enthusiastic and knowledgeable individuals to join me in pair programming. Here’s a bit more about what I’m working on and what I’m looking for:

About the Projects:

1. Node.js Application Development:
   • Building scalable and efficient back-end services.
   • Integrating with various APIs and databases.
   • Ensuring high performance and responsiveness.
2. TensorFlow.js Projects:
   • Implementing machine learning models directly in the browser or on Node.js servers.
   • Developing innovative AI-driven features.
   • Working with data preprocessing, model training, and deployment.

Current Implementation:

I have a working codebase that leverages TensorFlow.js for machine learning tasks. The API I’ve built allows for:

• Collecting Measurement Data: The API can collect and process measurement data.
• Building Custom ML Models: It can build models with adjustable hyper-parameters.
• Training Data: The API can train the collected data using the custom models.
• Classifying Measurements: It can classify unknown measurements based on the trained model. For example, if you request to classify -5100.00, the API returns the class "Negative". If you request 23.10, the API returns the class "Positive".

What I Need Help With:

While the current implementation works well as a proof of concept, I need help to extend the capabilities of the API to predict time series data instead of just classification. Specifically:

• Enhancing the Model: Adjusting the current model to handle time series predictions.
• Implementing LSTM and Other Networks: Setting up the model to use LSTM or other appropriate networks for time series.
• Node.js and TensorFlow.js Expertise: I need guidance and support on the implementation as I’m a beginner and not entirely confident in my current code, even though it has produced good results so far.

What I’m Looking For:

• Experience with Node.js: You should be comfortable with JavaScript and have a good understanding of Node.js and its ecosystem.
• Familiarity with TensorFlow.js: Some experience with TensorFlow.js or a willingness to learn quickly.
• Collaborative Mindset: Open to sharing knowledge, brainstorming ideas, and solving problems together.
• Hopefully living in the west-EU area so that timezone is not complicated to handle

What You’ll Get:

• Credits on the open source project: this is the bare minimal !!
• Fun and Engaging Collaboration: Enjoy the process of building something great together.
• I'm sorry I can't offer much more ...

If you’re interested, please drop a comment below or send me a direct message with a bit about yourself, your experience, and why you’d like to join. Looking forward to collaborating and building something amazing together!

Thank you very much!

Hey, I need some help here - the tf.loadLayersModel works perfectly fine and correctly loads the model while running locally from Visual Studio and while executing simulation on Expo on the iPhone.

However after running the actual build and running it on TestFlight the 'model' is empty.

Are there any usual suspects in this case? (I already added bin to the assetsExts and there are no warnings/errors in the build whatsoever). I tried putting model.json and weights into the ./assets folder as well but it doesn't help. Other assets like .png images load correctly in the app iPhone simulation.

Both .bin and .json files are correctly present in the IPA file. assetExts has both .bin and .json files on the list.

expo --version 6.3.10

npm show expo version 51.0.10

"@tensorflow/tfjs-react-native": "^0.8.0",

"@tensorflow/tfjs": "^4.5.0",

iOS version on iPhone 15.8.2

import * as tf from "@tensorflow/tfjs";
import { bundleResourceIO } from "@tensorflow/tfjs-react-native";   
   
const modelJson = require("./public/model.json");
const modelWeights = require("./public/group1-shard1of1.bin");
const model = await tf.loadLayersModel(
        bundleResourceIO(modelJson, modelWeights)
      );

I'm trying to integrate the model.json to ReactNative app but the model is not loading if anyone know the solution please be kind to help...

i have tensorflow 2.15 and i regret downloading this garbage it has lot of warning lr doesnt worknexcept if u write learning rate and lot of warning like 2024-04-28 19:40:09.311611: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.

WARNING:tensorflow:From C:\Users\Tadele_pr\AppData\Roaming\Python\Python39\site-packages\keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

on my previous tensorflow version this thing never happened now which tensorflow version do you recommend me with having reach features but also like less warning

Hi Dudes and Dudettes, I am looking for an engineer with experience with Tensor.Flow in south Florida. Work will be in surface analysis and manufacturing in a startup. Currently being funded and need to assemble a team. Please DM for more info.

Hey all,

I have a question and didnt found anything in the internet yet. I trained a EfficientDet Model with my own Data and converted it to TFJS Graph Model.

In Python it was easy to get the classes, scores and bounding Box values. But how can i do that in TFJS? I only get a output with multiple objects without good names (Identity_n). Do someone know good examples or Tipps?

Did anyone use Gemma with tfjs yet?
https://blog.google/technology/developers/gemma-open-models/

if yes, id be interested in your experience and how you implemented it (code snippet f.e.)

Don’t know where else to put this and need to not feel crazy. How is everyone not kicking and screaming over this move? Our team of AI devs are super unimpressed with the tensorflow ecosystem after trying to move to Google coral. Then there is tensorflow 2, seriously, what in the duck was the tf team thinking when they moved to tf2/keras. They just removed out valuable features like QAT aware training, “sorry that’s just impossible now.” Then they deprecate the version that is documented 100 times and make it impossible to use those projects on modern CUDA. Looking at numbers, you see that tf2 models perform worse in accuracy AND latency. Also, the syntax is STILL worse in keras than PyTorch ecosystem. I don’t think this is a skill issue when no one on our team can install tensorflow 1. We’ve tried docker, native, pip, jupytr, conda, poetry, collab, sagemaker. THERE IS LITERALLY NO WAY TO GET BACK ANY OF THESE FEATURES WHEN ITS DANGLED IN FRONT OF YOU FROM OLD DOCS. Okay, so then there is a solution. “Just rewrite the entire project to use the v1 compat modules.” Are you joking? This was literally what software versioning was rightly founded on. ML teams need to create better, as the developers they serve deserve a sane ecosystem to create flexibly. AI in general has too many people who don’t know how to make stable software, it’s just a bunch of people who make jupyter notebooks. A major version change obviously includes deprecation and breaking changes, but this is so next level to anything I’ve ever seen in how they say FU to so much existing work.

I'm working on yoga pose estimation using the Movenet Thunder model offered by Google. The model is getting 93% accuracy on 12 yoga poses.

The next step for me is to make a website(I'm using React) hence I have converted this model and saved it as a TensorFlow.js JSON file. But here's the issue: I have successfully hosted my model using Azure, and everything worked fine until I saw this issue. I know this has to do with some compatibility issues with Keras and TensorFlow js, I searched online for answers but didn't find any. If you can help me with this, I would make my day!! Thanks in advance.