8000 feat: improved csv export by mertmit · Pull Request #11429 · nocodb/nocodb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: improved csv export #11429

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 21, 2025
Merged

feat: improved csv export #11429

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c3dcb38
feat: move csv export to background jobs
mertmit May 20, 2025
e472e40
feat: data export clean up processor
mertmit May 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 75 additions & 73 deletions packages/nc-gui/components/smartsheet/toolbar/ExportSubActions.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,83 +1,95 @@
<script setup lang="ts">
import type { RequestParams } from 'nocodb-sdk'
import { ExportTypes } from 'nocodb-sdk'
import { saveAs } from 'file-saver'
import * as XLSX from 'xlsx'

const { $api, $poller } = useNuxtApp()

const { appInfo } = useGlobal()

const isPublicView = inject(IsPublicInj, ref(false))

const fields = inject(FieldsInj, ref([]))
const selectedView = inject(ActiveViewInj)!

const baseStore = useBase()
const { base } = storeToRefs(baseStore)
const urlHelper = (url: string) => {
if (url.startsWith('http')) {
return url
} else {
return `${appInfo.value.ncSiteUrl || BASE_FALLBACK_URL}/${url}`
}
}

const { $api } = useNuxtApp()
const handleDownload = async (url: string) => {
url = urlHelper(url)

const meta = inject(MetaInj, ref())
const isExpired = await isLinkExpired(url)

const selectedView = inject(ActiveViewInj)
if (isExpired) {
navigateTo(url, {
open: navigateToBlankTargetOpenOption,
})
return
}

const { activeNestedFilters: nestedFilters, activeSorts: sorts } = storeToRefs(useViewsStore())
const link = document.createElement('a')
link.href = url
link.style.display = 'none' // Hide the link

const isExportingType = ref<ExportTypes | undefined>(undefined)
document.body.appendChild(link)
link.click()
document.body.removeChild(link)
}

const isExporting = ref(false)

const exportFile = async (exportType: ExportTypes) => {
let offset = 0
let c = 1
const responseType = exportType === ExportTypes.EXCEL ? 'base64' : 'blob'
try {
if (isExporting.value || !selectedView.value.id) return

isExportingType.value = exportType
isExporting.value = true

try {
while (!isNaN(offset) && offset > -1) {
let res
if (isPublicView.value) {
const { exportFile: sharedViewExportFile } = useSharedView()
res = await sharedViewExportFile(fields.value, offset, exportType, responseType, {
sortsArr: sorts.value,
filtersArr: nestedFilters.value,
})
} else {
res = await $api.dbViewRow.export(
'noco',
base.value?.id as string,
meta.value?.id as string,
selectedView?.value.id as string,
exportType,
{
responseType,
query: {
fields: fields.value.map((field) => field.title),
offset,
sortArrJson: JSON.stringify(sorts.value),
filterArrJson: JSON.stringify(nestedFilters.value),
encoding: exportType === ExportTypes.EXCEL ? 'base64' : undefined,
},
} as RequestParams,
)
}

const { data, headers } = res

if (exportType === ExportTypes.EXCEL) {
const workbook = XLSX.read(data, { type: 'base64' })

XLSX.writeFile(workbook, `${meta.value?.title}_exported_${c++}.xlsx`)
} else if (exportType === ExportTypes.CSV) {
const blob = new Blob([data], { type: 'text/plain;charset=utf-8' })

saveAs(blob, `${meta.value?.title}_exported_${c++}.csv`)
}

offset = +headers['nc-export-offset']

setTimeout(() => {
isExportingType.value = undefined
}, 200)
let jobData: { id: string }

if (isPublicView.value) {
if (!selectedView.value.uuid) return

jobData = await $api.public.exportData(selectedView.value.uuid, exportType, {})
} else {
jobData = await $api.export.data(selectedView.value.id, exportType, {})
}

message.info('Preparing CSV for download...')

$poller.subscribe(
{ id: jobData.id },
async (data: {
id: string
status?: string
data?: {
error?: {
message: string
}
message?: string
result?: any
}
}) => {
if (data.status !== 'close') {
if (data.status === JobStatus.COMPLETED) {
// Export completed successfully
message.info('Successfully exported data!')

handleDownload(data.data?.result?.url)

isExporting.value = false
} else if (data.status === JobStatus.FAILED) {
Comment on lines +59 to +82
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Unsubscribe from poller to avoid endless replicas

$poller.subscribe returns an unsubscribe handle, but it’s ignored.
Leaving subscriptions active after navigation/spa reloads keeps sockets open and triggers duplicate callbacks.

const unsub = $poller.subscribe(…)
onUnmounted(unsub)

or if the component persists, call unsub() inside the completion/failed branch once the job ends.

🤖 Prompt for AI Agents 
In packages/nc-gui/components/smartsheet/toolbar/ExportSubActions.vue between
lines 59 and 82, the subscription to $poller is not unsubscribed, causing
potential memory leaks and duplicate callbacks. Fix this by capturing the
unsubscribe function returned by $poller.subscribe and either register it with
onUnmounted to clean up on component unmount or call the unsubscribe function
explicitly inside the completion and failure branches to stop polling once the
job finishes.

message.error('Failed to export data!')

isExporting.value = false
}
}
},
)
} catch (e: any) {
isExportingType.value = undefined
message.error(await extractSdkResponseErrorMsg(e))
isExporting.value = false
}
}
</script>
Expand All @@ -89,20 +101,10 @@ const exportFile = async (exportType: ExportTypes) => {

<NcMenuItem v-e="['a:download:csv']" @click.stop="exportFile(ExportTypes.CSV)">
<div class="flex flex-row items-center nc-base-menu-item !py-0 children:flex-none">
<GeneralLoader v-if="isExportingType === ExportTypes.CSV" size="regular" />
<GeneralLoader v-if="isExporting" size="regular" />
<component :is="iconMap.ncFileTypeCsvSmall" v-else class="w-4" />
<!-- Download as CSV -->
CSV
</div>
</NcMenuItem>

<NcMenuItem v-e="['a:download:excel']" @click.stop="exportFile(ExportTypes.EXCEL)">
<div class="flex flex-row items-center nc-base-menu-item !py-0 children:flex-none">
<GeneralLoader v-if="isExportingType === ExportTypes.EXCEL" size="regular" />
<component :is="iconMap.ncFileTypeExcel" v-else class="w-4" />

<!-- Download as XLSX -->
Excel
</div>
</NcMenuItem>
</templa 6D40 te>
4 changes: 3 additions & 1 deletion packages/nc-gui/plugins/api.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import type { Api } from 'nocodb-sdk'

Comment on lines +1 to +2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Loss of precise typing – we now fall back to Api<any>

ReturnType<typeof createApiInstance> preserved the real generic argument (SecurityDataType) so $api methods returned strongly-typed payloads.

With

$api: Api<any>

we lose compile-time validation of:

  • request/response bodies
  • union discriminations (e.g. ExportTypes)

Unless there was a cyclic-import problem, prefer keeping the specific type:

-import type { Api } from 'nocodb-sdk'
+import type { Api } from 'nocodb-sdk'
+import type { createApiInstance } from '@/composables/useApi' // adjust path

-  $api: Api<any>
+  $api: ReturnType<typeof createApiInstance>

(or expose a typed Api<SecurityData> alias from useApi).

Restoring strong typing prevents silent breakages when backend signatures change.

Also applies to: 12-13

🤖 Prompt for AI Agents 
In packages/nc-gui/plugins/api.ts around lines 1 to 2 and also lines 12 to 13,
the type of $api is currently set to Api<any>, which loses the precise generic
typing and disables compile-time validation of request/response bodies and
discriminated unions. To fix this, replace Api<any> with the specific generic
type that was originally inferred by ReturnType<typeof createApiInstance>, such
as Api<SecurityDataType>, or alternatively expose and use a typed Api alias from
useApi. This will restore strong typing and prevent silent breakages from
backend changes.

const apiPlugin = (nuxtApp) => {
const { api } = useApi()

Expand All @@ -7,7 +9,7 @@ const apiPlugin = (nuxtApp) => {

declare module '#app' {
interface NuxtApp {
$api: ReturnType<typeof createApiInstance>
$api: Api<any>
}
}

Expand Down
21 changes: 0 additions & 21 deletions packages/nocodb/src/controllers/data-alias-export.controller.spec.ts
View file
Open in desktop

This file was deleted.

116 changes: 0 additions & 116 deletions packages/nocodb/src/controllers/data-alias-export.controller.ts
View file
Open in desktop

This file was deleted.

23 changes: 0 additions & 23 deletions packages/nocodb/src/controllers/public-datas-export.controller.spec.ts
View file
Open in desktop

This file was deleted.

Loading
Loading
0