Questions tagged [mlrun]

open source helper/framework based on Python for speedup delivery of ML/AI to the production (support MLOps with featurestore concept), main focus on kubernetes solution. Use this tag for questions about platform MLRun (e.g. MLRun CE, MLRun paid version) or libraries from this solution (e.g. v3io, v3io stream, storey).

MLRun is an open source helper/framework based on Python for speedup delivery from ML/AI experiment phase to production with focus on minimalizing development effort (agnostic approach without focus on a specific AI/ML topic).

It supports MLOps with featurestore concept, including continuous training and continuous monitoring also. It is based on K8s with ability to support scalability of AI/ML in on-prem and cloud (support of main three major cloud providers).

Sample of python code:

import mlrun 
import mlrun.feature_store as fstore
    
# create project 'test' 
project = mlrun.get_or_create_project("test", context="./")

# add feature set (part of feature store)
transaction_set = fstore.FeatureSet("transaction", entities[fstore.Entity("trid")])

# ingest data
fstore.ingest(transaction_set, bank_trn, infer_options=fstore.InferOptions.default())

More information

  • License 'Apache-2.0 license'
  • Official web see
  • Git web see
  • Documentation web see
59 questions
0
votes
1 answer

v3io, DateTime "GetItems: invalid value of JSON request parameter"

I got this error during filtering of values in NoSQLTarget via v3io: ... raise HttpResponseError("Request failed with status {0}: {1}".format(self.status_code, self.body)) v3io.dataplane.response.HttpResponseError: Request failed with status 400:…
XiongChan
  • 90
  • 12
0
votes
1 answer

MLRun, ErrorMessage, No space left on device

I got this error during ingest data to FeatureSet: Error - Failed to save aggregation for /k78/online_detail/nosql/sets/on line_detail/0354467518.ed74fc2b Response status code was 400: b'{\n\t"ErrorCode": -28,\n\t"ErrorMessage": "No space left on…
JIST
  • 1,139
  • 2
  • 8
  • 30
0
votes
1 answer

MLRunNotFoundError: there are no offline targets for this feature set

I got this error during FeatureVector call: ... File "C:\Python\emptyVectore\venv\lib\site-packages\mlrun\feature_store\retrieval\base.py", line 105, in start return self._generate_vector( File…
XiongChan
  • 90
  • 12
0
votes
1 answer

Error "value of key _fn0 is None" during data ingest

I got this mistake in MLRun, during ingest values to the FeatureSet: > 2023-05-20 13:11:40,669 [info] loaded project my-project7xx from ./ and saved in MLRun DB > 2023-05-20 13:11:53,640 [error] For {'_1': 374, '_2': 886, '_3': 989, '_4': 191, '_5':…
JIST
  • 1,139
  • 2
  • 8
  • 30
0
votes
1 answer

Issue with ODBC in MLRun Jupyter (installation SSL, Kerberos, ODBC)

I would like to access from MLRun jupyter to bigdata platform via ODBC with SSL and Kerberos and use this sample/easy code. import pyodbc import pandas as pd conn=pyodbc.connect("DSN=TST", autocommit=True) crsr = conn.cursor() crsr.execute('show…
XiongChan
  • 90
  • 12
0
votes
1 answer

Issue with mount persistent storage in workflow, File not found

When I used fn.apply(mlrun.mount_v3io()) in MLRun function, it worked correctly and I can access to files in mount persistent storage. import mlrun ... fn = mlrun.import_function("test-function.py") # use mount_v3io() for iguazio…
XiongChan
  • 90
  • 12
0
votes
1 answer

v3io, Limit for amount of files

Do you know limits or formula, how to calculate amount of files in v3io? It probably has relation to count of data nodes in cluster and size of memory on data server nodes. I tried to use 50 and also 80 millions of files in v3io and it performs…
XiongChan
  • 90
  • 12
0
votes
1 answer

Unexpected duration for v3io content delete

When I deleted content of NoSqlTarget (key-value storage) in MLRun/v3io via standard command line utility such as: rm -rf /v3io/project/test-featureset/ It took approx. 1 hour for 1 million of files and in case of 20 millions of files it was near…
JIST
  • 1,139
  • 2
  • 8
  • 30
0
votes
1 answer

igztop, issue with unsufficient permissions in webshell

I cannot run command igztop in MLRun WebShell. I got this mistake: xiong @ iguazio-system 13:26:39 ] ~ $ igztop No resources found. Does the Service Account running this service have sufficient permissions? Use 'igztop --check-permissions' to find…
XiongChan
  • 90
  • 12
0
votes
1 answer

MLRunPreconditionFailedError: 412 Client Error, "API is waiting for migrations to be triggered"

I got this error: MLRunPreconditionFailedError: 412 Client Error: Precondition Failed for url: http://localhost:8080/api/v1/projects: Failed creating project tutorial- details: MLRunPreconditionFailedError('API is waiting for migrations to be…
JIST
  • 1,139
  • 2
  • 8
  • 30
0
votes
2 answers

Issue, iteration cross KV (NoSql/Redis) Target

I would like to iterated cross KV (NoSql,Redis) Target in MLRun without specification of primary key(s) also, but I saw only ability to get specific KV item(s) based on specific key(s). I have to use only unique items, not duplicit items. I chose…
JzD
  • 22
  • 11
0
votes
1 answer

Is it possible to remove specific items from Parquet and NoSQL Targets?

Is it possible to remove specific items from Parquet and NoSQL Targets in MLRun? I did not find relevant methods and I checked FeatureStore, ParquetTarget and NoSQLTarget. I only saw ability to remove whole featureset from metastore (from DB)…
XiongChan
  • 90
  • 12
0
votes
1 answer

Missing webapi parameter or V3IO_API environment variable

I got the exception: File "C:\Python\ingestKVdatetime\venv\lib\site-packages\mlrun\datastore\targets.py", line 1155, in get_table_object V3ioDriver(webapi=endpoint), File "C:\Python\ingestKVdatetime\venv\lib\site-packages\storey\drivers.py",…
JIST
  • 1,139
  • 2
  • 8
  • 30
0
votes
1 answer

Sklearn-classifier, issue with freez (pod pending in K8s)

I got freez of Sklearn-classifier in MLRun (the job is still running after 5, 10, 20, ... minutes), see log output: 2023-02-21 13:50:15,853 [info] starting run training uid=e8e66defd91043dda62ae8b6795c74ea DB=http://mlrun-api:8080 2023-02-21…
JIST
  • 1,139
  • 2
  • 8
  • 30
0
votes
1 answer

Issue with MLRun Spark service start and impact all Jupyter notebooks

I reconfigured Spark infrastructure in K8s (as part of MLRun/Iguazio platform) and after that, I got a lot of issues in level of services: Spark service (with information Failed) All jupyter notebooks (with information Failed dependencies) and…
JIST
  • 1,139
  • 2
  • 8
  • 30