Questions tagged [azure-notebooks]
78 questions
0
votes
1 answer
Azure Workbook - Authenticate to Azure Resources
enter image description here
Having an issue with this spot, 'Authentication to Azure Resources' not sure where I should define workspace. I thought that was covered in line 2.
I tried adding the workspace ID
0
votes
1 answer
Merging a Spark DF using INSERT INTO
I have an existing table, which I'd like to append two columns to. I create a Spark dataframe:
spark_df = spark.createDataFrame(df)
Then I'd like to use MERGE INTO as so:
spark.sql(f"""MERGE INTO x.y AS m
USING {spark_df} AS s
ON m.id = s.id
WHEN…
pymat
- 1,090
- 1
- 23
- 45
0
votes
1 answer
Create table and simultaneously replace coulmn contents with row number
I have the following statement:
CREATE TABLE result2 AS
SELECT t.*
FROM sample1 as t
CROSS JOIN (select * from range(0,10)) v(i);
select * from result2;
and I'd like to have in the created table (result2), which contains the column 'id' from…
pymat
- 1,090
- 1
- 23
- 45
0
votes
1 answer
Using CROSS JOIN SEQUENCE to produce large duplicated tables
I have the following in an Azure notebook (databricks sql):
CREATE TABLE my_new_big_table AS
SELECT t.*
FROM my_table t
CROSS JOIN VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10) v(i);
This duplicates the my_table, 10 times, however how can I…
pymat
- 1,090
- 1
- 23
- 45
0
votes
1 answer
Duplicate SQL table to creat large dummy table
I have an SQL table already, the format of which I just want duplicating for testing purposes. I tried the following:
DECLARE @Counter AS INT(1)
WHILE ( @Counter <= 10)
BEGIN
CREATE TABLE my_new_big_table
AS (SELECT * FROM my_table)
SET…
pymat
- 1,090
- 1
- 23
- 45
0
votes
1 answer
Lookup activity output in array
i have a lookup activity in ADF which retrieves data from table like below
{
"count": 1,
"value": [
{
"Notebook_path": "/Users/Name@work.com/Project/ingestion/Cust1/ADT"
}
],
"effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime…
Darkmaster
- 29
- 1
- 3
0
votes
1 answer
How import multiple notebooks into a new notebook using %run command
Can anyone suggest the syntax to import multiple notebooks using %run command in Databricks?
Right now, am importing through seperate run commands for every notebook like below:
%run ./Notebook1
%run ./Notebook2
%run ./Notebook3
I would like to know…
kiruba
- 129
- 5
0
votes
1 answer
Synapse Notebook throws timeout error while connecting to AWS RDS SQL Server
I am working in the Synapse Workspace and trying to connect to AWS RDS from the Synapse Notebook.
Whenever I try to connect, it throws the below timeout error -
The TCP/IP connection to the host my-host, port 1433 has failed.
Error: "connect timed…
Sanket Kelkar
- 129
- 2
- 9
0
votes
0 answers
errorCode": "6000" - azure synapse running the pipeline
I have error when running a pipeline in azure synapse. If I execute the synapse notebook manually it works good (readind and writing). But when I call the name notebook from the ForEach activity inside the pipeline it fails to run. I have the…
Greencolor
- 501
- 1
- 5
- 16
0
votes
1 answer
Can I get pipeline parameters in Synapse Studio Notebook dynamically without predefining parameter names in parameter cell?
In the Pipeline that triggers the Notebook, I'm passing some base parameters in, as below.
Right now, all the materials I read instructed me to declare variables inside the Notebook parameter cell with the same names, as below.
Is there a way that…
xiexieni9527
- 111
- 7
0
votes
0 answers
Geting error Caused by: com.databricks.NotebookExecutionException: FAILED
I am trying to run the below notebook through databricks but getting the below error. I have tried to update the notebook timeout and the retry mechanism but still no luck yet.
NotebookData("/Users/mynotebook",9900, retry=3)
]
res =…
SeleniumUser
- 4,065
- 2
- 7
- 30
0
votes
1 answer
Get modified date of blobs in Synapse Notebooks
I have, in one Storage Account, a container with folders and subfolders. In one of them, like container foo and path fee/fii, i have a large amount of files. I want to get the last modified date of every blob in my Notebook. I cannot solve this…
Omar
- 1,029
- 2
- 13
- 33
0
votes
2 answers
How to time tigger a python script in the Azure ML notebooks
Hi I am currently working on a small image classification project where the model classifies whether the image contains potholes or not. For this section i have wrote the python script, and this script needs to be triggered at scheduled time. I…
Wishva
- 25
- 7
0
votes
2 answers
Getting error while running the configuring MSTICPY on Azure sentinel notebook
I am using Azure sentinel notebook for threat intelligence. While trying to configure msticpy for it to connect to Azure sentinel but getting 'Value error'. Following is the code that I am using :
from msticpy.config import MpConfigEdit
import…
Lopa
- 51
- 6
0
votes
1 answer
How to run a robot framework script from azure databricks notebook?
Is it possible to run robot framework test suite using azure data bricks notebook?.
I have a set of robot framework test suite, that uses database library, Operating System library etc.
In my local machine, I install python, pip install all…
